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i.  INTRODUCTION 


An  engineer  would  be  rather  foolish  and  risk  his  reputation  if  he  assigned  a  definite  value  to 
the  system  reliability  of  a  missile  weapon  system  based  on  a  few  firings.  He  will  estimate  the 
system  reliability,  though,  and  ascertain  with  a  certain  degree  of  confidence  that  his  estimate  is 
within  a  certain  interval.  The  degree  of  confidence  expresses  his  state 
of  mind  and  reflects  the  extent  of  misjudgement  he  wishes  to  avoid;  thus  it  is  a  reflection  of  his 
consersatism.  If  a  number  is  ass.gned  to  this  confidence,  it  becomes  a  confidence  level  and 
specifies  the  percentage  of  the  statements  that  he  expects  wili  be  correct.  At  this  stage  of  the 
anaksis.  the  engineer  will  determine  his  trade-off  point  between  the  desired  confidence  level 
and  the  quantity  of  testing  (in  this  case,  the  number  of  missile  launchings)  based  on  available 
time  and  money.  Thus,  he  can  establish  a  quantitative  interval  on  his  estimates  which  reflects 
his  professional  confidence  and  represents  his  monetary  limitations.  He  has  2lso  helped  to 
alleviate  an  inherent  flaw  in  higher  management  decision  making,  that  of  misinterpretation  of 
data  due  to  a  lack  of  statistical  background,  which  contributes  to  possible  gross  errors  in 
judgement  since  intuition  cannot  be  used  as  a  safeguard.  He  has  also  presented  the  decision 
maker  with  a  knowledge  of  the  risk  involved  and  of  the  limitations  of  his  information. 

Thus,  for  any  one-shot  weapon  system  or  explosive  device  where  the  test  data  are  limited, 
the  best  that  the  engineer  can  say  of  the  true  system  reliability,  or  some  other  form  of  output 
measurement,  is  that  it  lie*  within  some  confidence  limits  with  a  certain  degree  of  confidence. 
These  limits  with  the  associated  degree  of  confidence  ha.c  io  oe  clearly  defined,  based  on  the 
many  variables  and  limitations  of  the  unit  under  test,  and  the  test  itself,  in  order  to  provide  the 
most  informative  statements  based  on  tiic  test  results.  When  working  with  confidence 
measurements,  two  basic  principles  are  common;  ( 1 )  for  a  given  quantity  of  test  data,  the 
higher  the  confidence  level,  the  larger  the  confidence  interval  and  vice  versa,  and  (2)  the  greater 
the  quantity  of  test  data  the  narrower  the  confidence  interval  for  any  specified  confidence 
level. 

These  concepts  provide  the  underlying  framew  ork  for  determining  a  value  for  the  output  of 
a  system  and  quantifying  the  worth  of  that  value.  The  ultimate  output  of  any  system  is  the 
performance  of  some  intended  function,  usually  referred  to  as  the  mission  with  respect  to 
weapon  systems.  The  term  used  to  describe  the  overall  capability  of  a  system  to  accomplish  its 
mission  is  system  effectiveness,  [80,  p.  I]* 

In  1965  a  special  committee  known  as  the  Weapon  System  Effectiveness  Industry  Advisory 
Committee  ( WSE1  AC),  chartered  through  Department  of  Defense  Directive,  published  their 
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reports.  1  hese  reports  addressed  the  problem  of  developing  a  standard  technique  to  apprise 
management  of  current  and  predicted  weapon  system  effectiveness  during  all  phases  of  the 
weapon  system  lifecycle.  WSLS  AC  states  that  effectiveness  is  a  function  of  system  availability, 
dependability,  and  capability.  J  29.  p.  42],  [2]  The  result  of  the  expression  of  these  parameters 
in  the  system  effectiveness  model  is  used  to  provide  decision  information  to  high  level 
management  with  a  required  high  degree  of  confidence  in  the  model,  thus  providing  a  measure 
of  a  system  worth. 

It  is  sometimes  necessary  to  modify  the  basic  guidance  provided  by  WSElAC  in  order  to 
model  a  particular  system  since  effectiveness  is  influenced  by:  { 1 )  the  way  the  equipment  is 
designed  and  built.  (2)  used  and  maintained.  (3)  rules  of  engagement,  (4)  fiscal  control  and  (5) 
many  other  administrative  policy  decisions.  With  this  in  mind,  a  general  system  effectiveness 
definition  for  a  one-shot  device  such  as  a  missile  is:  “System  effectiveness  is  the  probability 
that  the  system  (missile)  will  operate  successfully  (kill  the  target)  when  called  upon  to  do  so 
underspecified  conditions.”  [80,  p.  5]  This  definition  is  also  structured  to  give  an  effectiveness 
expression  for  the  respective  system.  As  an  example,  the  system  effectiveness  expression  fora 
Forward  Edge  of  the  Battle  Area  (FEBA),  man-portable,  shoulder  fired,  air  defense  missile 
system  might  be: 

E,  R,  x  Ru  x  L-m  x  r del 

E,  =  System  Effectiveness 

R,  —  FreUSe  Reliability  Ritoelpi^  X  Rfie,d  cm  non  mcni 

R,  =  Weapon  Reliability  =  Racine  x  R  fire  X  Runhcad  detonalicn  (Rfu/c  X  R»irhcid) 

Lm  —  Missile  Lethality 

Pdc  =  Probability  of  Detection,  E'.uiuati'  n  and  Transfer. 

The  majority  of  terms  in  this  expression  are  determined  from  actual  tests  on  components  of 
the  system  or  the  entire  system  itself.  The  system  level  tests  are  structured  such  that  resuits  are 
applicable  to  the  specific  phases  (terms)  in  the  reliability  expressions  of  the  system 
effectiveness  model.  For  convenience  and  ease  of  derivation,  the  phases  arc  assumed 
independent  and  test  results  arc  usually  expressed  as  binomial  results  or  success  ratios  taken 
from  attribute  data.  By  definition,  the  point  estimates  of  each  reliability  phase,  obtained  from 
reliability  scoring  of  test  data,  are  combined  as  a  scries  product  to  produce  weapon  and  preusc 
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reliability  at  a  required  confidence  level.  At  this  point,  an  inherent  problem  arises  in 
generating  the  necessary  reliability  values  at  a  predesignated  required  confidence. 

One  of  the  acceptable  ways  of  determining  system  reliability  with  confidence  is  by 
combining  the  point  estimates  oi  the  respective  system  reliability  scoring  phases  by  a  single¬ 
thread  approach.  This  approach,  which  successively  discards  failures  based  on  equal  sample 
tests,  enables  a  confidence  level  to  be  placed  on  the  binomial  system  reliability  estimate.  An 
example  of  this  approach  for  a  three  phase  reliability  expression  is: 


18  15  12  _  12 

20  X  18  X  15  20 

This  approach  does  not  represent  the  real  world  conditions  due  to  time,  money,  and  realistic 
test  programs  which  yield  unequal  sample  sir  s  for  each  phase.  Test  programs  are  structured 
such  that  each  respective  type  of  test  is  independent  of  another  type  and  each  type  corresponds 
to  a  particular  reliability  scoring  phase  or  results  of  a  combination  of  phases.  These  tests 
and/or  reliability  phases  represent  respective  distributions  which  are  generally  unknown  as 
well  as  describing  a  system  with  an  unknown  distribution. 


Presently  a  problem  exists  in  being  able  to  place  a  confidence  level  on  missile  weapon 
reliability.  This  paper  suggests  a  method  for  solving  the  problem.  This  method  provides  the 
means  for  estimating  the  system  reliability  lower  confidence  limit  with  specified  confidence 
level  from  reliability  phase  scoring  data  derived  from  system  and  subsystem  test  results  and 
yields  a  quick  and  good  approximation  to  a  more  rigorous  and  lengthy  analysis.  The  method 
does  not  require  any  assumptions  concerning  the  form  of  the  reliability  phase  distributions 
nor  their  test  sample  size. 


A.  METHOD 


Since  most  component  and  system  data  are  made  up  of  attributes  in  the  form  of  observable 
two  state  test  results  based  on  a  go/ no-go  criterion,  data  are  collected  in  the  form  of  binomial 
success  ratios.  These  ratios  are  characterized  as  success  probabilities,  ps,  on  the  components 

A  A 

of  the  system  or  R’s  when  concerning  the  system,  where  p  is  an  estimate  obtained  as  a 
probability  from  p  =  !25n  when  n  components  are  tested  and  d  of  them  fail,  so  that  n-d  complc  .* 
the  test  successfully,  likewise  concerning  system  test  results.  Thus  it  is  known,  as  in  the 
common  coin  tossing  example,  that  this  probability  estimate  has  a  binomial  distribution 
about  the  true  p  or  R. 
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With  the  necessary  attribute  test  data  collected,  the  reliability  of  the  system  is  represented  as 

a  A  A 

a  mathematical  unction  of  random  variables  of  the  form  R  (pi,  p^,  .  .  .  p„)  which 
is  representative  of  a  system  built  up  from  various  series  and  parallel  arrangements  of 
components.  In  this  case  the  reliability  system  estimate.  R.  may  be  taken  as  an  unbiased 

estimate  of  the  parameter  R(pi,  p; . pfl),  that  is.  the  system  reliability.  If,  as  in  the  case  of 

interest  here,  the  reliability  of  the  system  is  a  mathematical  function  of  system  reliability  phase 

scoring  estimates,  it  would  have  the  form  R(n.r: . r„)and  would  be  taken  as  the  unbiased 

estimate  of  the  system  reliability  R  (n.  r; . rs).  At  this  point,  the  question  arises  as  to 

whether  the  results  of  any  test  or  phase  are  dependent  upon  the  results  of  other  tests  cr  phases. 

The  answer  is  intuitively  yes.  and  should,  if  possible,  be  taken  into  account  in  any  reliability 

prediction  and  the  determination  of  confidence  limits.  However,  in  this  paper,  it  will  be  '* 

assumed  that  all  test  results  are  independent,  thus  uncorrelated,  in  order  to  simplify  the 

discussion  and  keep  the  algebraic  computations  to  a  minimum. 

Since  one  of  the  prime  objectives  of  this  paper  is  to  present  a  relatively  simple  method  of 
determining  an  estimate  for  the  lower  confidence  limit  on  system  reliability,  the  method  for 
generating  a  one-sided  tolerance,  or  control,  limit  in  construction  of  statistical  quality  control 
charts  was  adopted.  This  limit  is  given  by  the  quantity  R-Kor  with  the  property  that  the 
probability  is  7  that  at  least  a  proportion  ( 1-/3 )  of  the  distribution  will  be  contained  within  the 

A 

interval  R-Kor  and  plus  one.  The  quantity  K  is  called  a  tolerance  factor  and  is  a  function  of  7 
and  /3. 

Normally  the  assumption  of  normality  of  system  performance,  based  on  the  central  limit 
theorem,  is  applied,  but  this  is  applicable  only  when  system  performance  is  the  sum  of  the 
effects  of  many  component  or  subsystem  test  results,  or  reliability  scoring  phases,  with  no 
single  one  having  a  dominant  variance.  Since  sample  sizes  are  usually  not  large  enough  for  the 
central  limit  theorem  to  apply,  the  indiscriminate  assumptions  of  normality  could  lead  to 
erroneous  conclusions.  Based  on  this,  the  estimate  of  system  variance  will  b:  derived  from  the 
generation  of  system  moments  which  neither  requires  assumptions  concerning  the  form  of  the 
scoring  phase  distributions  r.or  test  sample  sizes  associated  with  eac«.  phase.  It  aiso  allows  an 
analysis  to  be  made  of  the  importance  of  each  phase  variable  through  the  examination  of  the 
magnitude  of  its  partial  derivative.  This  method  is  commonly  referred  to  as  statistical  error 
propagation  or  the  delta  method  and  is  developed  through  the  concept  of  expected  values  with 
a  Taylor  scries  exj  ansion  about  the  point  at  which  each  of  the  component  variables  takes  on 
its  expected  value,  or  first  moment. 

Since  independence  has  been  assumed  (it  need  not  be)  and  a  distribution-free  method  for 
obtaining  system  parameters  has  been  adopted.  thcCair.p-Meidcll  Inequality  for  determining 
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the  tolerance  tactor,  K,  will  be  used.  This  is  a  distribution-free  and  fairly  conservative  method, 
but  a  necessary  one  since  neither  the  phase  nor  system  distributions  are  known.  (This  method 
is  a  modified,  more  accurate  version  of  method  number.6  described  in  the  literature  review 
which  follows.) 

The  remainder  of  Chapter  I  gives  the  literature  review.  In  Chapter2  the  proposed  method  is 
formulated,  explained,  demonstrated,  and  compared.  Chapter  3  applies  the  proposed  method 
to  a  realistic  missile  weapon  system  in  its  development  life  cycle  phase.  It  presents  a 
comprehensive  explanation  of  the  testing  philosophy  and  reliability  phase  scoring  criteria  and 
methodology  involved  in  the  system  development.  Chapter  4  summarizes  the  results  and 
offers  recommendations. 

B.  LITERATURE  REVIEW 

A  comprehensive  literature  search  and  investigation  of  the  problem  was  made.  This  section 
reviews  the  investigation  and  presents  some  method  descriptions  and  comments  on  selected 
methods  which  are  deemed  sufficient  in  order  to  cover  the  broad  spectrum  of  solutions  that 
exist  in  the  literature,  which  might  be  applicable.  Through  this  investigation,  it  is  evident  that 
direct,  exacting  analytic  methods  for  the  problem  solution  are  not  feasible  due  to  the 
complexity  of  the  system  (the  mathematical  model  which  defines  its  reliability),  the 
nonavailability  of  a  computer  facility  with  “canned”  programs  for  the  solution  of  the  problem, 
and  not  enough  time  available  for  a  rigorous  solution  in  support  of  management  decisions. 

A  review  of  the  existing  literature  follows: 

1 .  A  method  for  determining  confidence  interval  estimation  of  the  reliability  of  multi- 
component  systems  using  component  test  data  was  developed  by  Johnson  [34]  using  the  exact 
multivariate  binomial  distribution.  Basic  to  this  method  is  the  dependence  of  the  reliability  of 
the  system  Tn  the  reliability  of  its  components  which  is  determined  by  the  structure  of  the 
system  such  that  the  reliability  of  the  system  can  be  expressed  as  a  known  function  of  the 
failure  rates  of  the  components.  The  presentation  assumes  an  ordering  of  the  N  points  of  the 
sample  space  of  the  vector  d  =  (di,  di, .  .  .,  dc),  where  d,  is  the  number  of  components  in  the 
sample  of  the  ilh  type  of  component  that  failed  when  tested.  Superscripts  designate  the 
ordering  of  the  sample  point.  Thus  with  the  above  assumptions,  the  construction  of  a  one-side 
confidence  interval  for  system  reliability  is  the  solution  of  the  following  nonlinear 
mathematical  programming  problem: 
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Minimize  the  objective  function 


R  =  R  (p!*  p2»  • ••»  pc) 
subject  to  the  constraints  that 


k 

c  r 

l 

n  (di 

r=l 

i=l  V1 

0  1 

>  1-Y 


where:  R(p)  =  the  reliability  function  for  the  particular  system  under  consideration 
n,  =  the  number  of  items  tested  of  the  ilh  type  of  component  in  the  system 

d,  =  the  number  of  failures  observed  in  the  ith  type  component 

k  =  the  order  index  of  d 
y  =  the  confidence  coefficient  (level) 

c  =  the  number  of  different  types  of  components. 


This  method  requires  a  high  capacity,  high  speed  digital  computer  and  reprogramming  for 
each  specific  problem  based  on  the  system  structure.  It  should  also  be  emphasized  that 
component  (subsystem)  iailure  rates  will  most  often  be  unknown  and  will  have  to  be  estimated 
from  test  results. 

2.  The  linearization  method  of  obtaining  confidence  intervals  for  system  reliability  based 
on  the  asymptotic  normality  of  the  maximum  likelihood  estimates  (MLE)  was  presented  by 
De  Cicco  [19]  and  further  enunciated  by  Rosenblatt  [61]  and  Johnson  [34],  This  method 
develops  as  follows: 

If  the  system  reliability  function  is  written  as 
R  “  R  P2>  •••»  Pc) 
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i 

\ 


A 

the  MLE  of  R,  R,  can  be  obtained  by  replacing  the  p’s(p,)  by  their  (its)  MLEp:  =  d,/nii  =  1,2, 

.  .  .,  c,  [36,  p.199]  where  d,  is  the  number  of  i"  type  components  that  failed  during  tests  and  n, 
is  the  number  of  items  tested  of  the  i!l'  type  of  component.  Here  p,  (the  M  LE  of  the  failure  rate 
of  the  ith  type  component)  has  an  asymptotic  normal  distribution  with  mean  p,  and  variance  pi 
(1  -  p,)/n,.  (24,  p.  1 33 1  It  also  follows  that  R  is  asymptotically  normal  with  mean  Rand  variance 
[57,  p.207 J 

var(Pl)' 

If  p,  is  substituted  for  the  unknown  parameter  p,  in  the  equation  for  a2,  approximate 
confidence  intervals  can  be  obtained  such  that 

P<«-  Z1+Y  “Y. 

2  ”2 

Thus  an  approximate  IOO7  per  cent  one-sided  lower  confidence  limit  for  R  can  be 
constructed  from  the  above. 

In  using  this  method  it  should  be  recognized  that  it  might  produce  confidence  intervals  that 
do  not  contain  R  thus  the  true  confidence  coefficient  might  be  considerably  smaller  than 
intended.  It  should  also  be  recognized  that  computations  for  determining  the  MLE  are  more 
difficult  than  for  determining  moments  and  that  the  M  LE  sometimes  yields  biased  estimates. 

3.  Shooman  [66]  addresses  the  question  of  approximations  and  bounds  on  system 
reliability  when  considering  a  system  of  n  components  (subsystem)  with  information  on  each 
component  reliability  but  little  information  on  their  interconnection.  The  essential  factors  for 
the  approaches  discussed  are:  the  component  hazards,  the  system  structural  model,  and  the 
dependence  of  the  components.  Fora  r-out-of-n  structure,  expressions  for  the  exact  reliability 
were  derived  using  the  binomial  distribution  for  independent  and  identical  components  or  by 
a  structural-model-app roach  if  the  components  were  dependent.  For  this  same  structure,  the 
Poisson  approximation  of  the  binomial  covering  the  ends  of  the  reliability  range  was  given. 
This  derivation  used  the  average  probabilities  of  success  and  failure  and  the  normal 
approximation  was  presented  to  cover  the  middle.  Bounds  on  series  structures  with  many 
elements  were  de  ved  using  the  failure  law  of  complex  equipment.  This  was  also  presented  as 
a  pessimistic  estimate  if  parallel  paths  were  present  in  the  system. 


II 


An  alternative  to  the  other  approaches  was  the  development  of  bounds  on  system  reliability 
through  the  application  of  cut-set  and  tic-set  analysis  when  a  more  complex  problem  was 
encountered  in  which  the  form  of  the  reliability  function  became  very  difficult  to  write  down. 
This  analysis  was  developed  from  the  properties  of  the  reliability  graph  for  any  system  not 
containing  dependent  failures. 

For  the  application  of  the  above  approaches,  it  is  expected  that  the  bounds  will  be  quite 
loose  and  that  care  and  judgement  be  applied  in  their  utilization. 

A  brief  discussion  of  interval  estimates  (confidence  interval)  was  presented  in  which  a 
component  hazard  rate  was  assumed  and  the  maximum- likelihood-estimate  (MLE)  was 
computed  for  system  mean,  variance,  and  reliability  function.  These  values  were  then  used  to 
form  a  confidence  interval  based  on  the  assumed  estimated  distribution. 

Another  method  was  to  calculate  the  variance  of  the  MLE  of  the  system  and  apply  the 
Tchebycheff  or  Gauss  inequality  to  predict  the  probability  that  a  random  variable  (system 
reliability  function)  of  unspecified  distribution  lies  in  an  interval. 

4.  Another  approach  for  determining  system  reliability  confidence  intervals  with  specified 
confidence  level  from  component  data  was  presented  by  Stolp  and  Welch  [75],  They  proposed 
an  extension  of  the  Neyman-Pearson  definition  of  a  confidence  interval  for  one  random 
variable  to  a  confidence  interval  for  a  function  of  several  random  variables. 

This  technique  involves:  (a)  a  point  estimator  of  the  total  weapon  reliability  as  a  function  of 
the  subsystem  point  estimators,  (b)  the  derivation  of  the  sampling  distribution  of  the  total 
weapon  statistic,  (c)  a  specified  overall  level  of  confidence  for  the  weapon  and  the  constraint 
that  this  overall  confidence  be  shared  equally  among  the  subsystems,  and  (d)  the  assumption 
that  all  subtier  levels  of  the  weapon  system  and  the  complete  system  itself  conform  to  the 
binomial  distribution.  The  authors  determined  the  unique  number  of  tests  to  be  run  with  zero 
failures  in  order  to  satisfy  the  requirements  and  extended  their  approach  and  philosophy  to 
the  case  of  one  or  more  failures.  The  mathematics  of  their  technique  are  based  on  the  relationship 

N 

i  =  1-C^  (for  the  zero  failure  case) 

which  says  that  reliability  and  lack  of  confidence  behave  in  the  same  manner.  Therefore,  fora 
series  configuration,  FIR,  =  Rs,  then  TI(  1  -C,)  =  1-C5, 


and 


P(f<k)  +  P(f>k)  =  I  (f  =  failure). 


which  sa\s: 

Therefore. 

and 

where:  R,  = 
N,  = 


P(R>R..)  +  P(R<R..)  =  I  (R  =  reliability). 

P( R>R„)  -  C  (confidence) 

P(R<R„)  -  1C  (lack  of  confidence), 
the  estimated  reliability  of  the  i"  subsystem 

the  number  of  tests  required  ol  the  ilh  subsystem  with  zero  failures  for  a 
specified  C,.  confidence 


1  -Ci  =  the  lack  of  confidence  of  the  estimated  reliability  for  the  ilh  subsystem 
C,  =  the  probability  of  (k)  or  fewer  failures  in  the  i'h  subsystem: 

P  [x<(k)l  =  I  ( x )  Rx  (1-R)N_x  . 

L  J  x=0 


Using  the  series  law  of  reliability  (product  rule),  the  following  reasoning  was  used  to 
combine  the  confidence  levels: 


if  R±Ni  =  1-CU,  i  =  1,2,  n 

and  R  =  1-C 
s  s 

where  R»  is  the  reliability  of  the  system  and  C»  is  the  confidence  level  of  the  estimated 
system  reliability  such  that 


R 

s 


Nr, 


R  NlR  N2  ...R  Ni  ...R  Nn 
12  i  n 


then 


i-cs  =  (i-c1)(i-c2)...(i-c1)...(i-cn). 


Upon  imposing  a  uniform  subsystem  confidence  constraint  they  get 

C  =  1-(1-C.)n 
s  i 
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or 


C,  =  l-Q-C  ) 
I  s 


1/n 


it  should  be  noted  that  if  the  number  of  subsystems  (components)  increases,  for  a  given 
weapon  confidence  level,  their  confidence  levels  decrease,  or  stated  another  way,  as  the 
number  of  subsystems  is  increased,  the  subsystem  confidence  level  decreases  thereby 
effectively  increasing  the  subsystem  reliability  value.  For  example,  20  subsystems  would  share 
a  90  per  cent  weapon  confidence  at  the  1 1  per  cent  confidence  level  and  25  subsystems  wouid 
share  at  approximately  8  per  cent. 


5.  A  method  for  determining  the  lower  reliab  lity  limit  of  the  system  with  respective 
system  confidence  level  w*s  proposed  by  Chance  [14],  This  method  is  based  on  the  joint 
probability  of  occurrence  for  the  system,  paced  by  the  system  reliability  estimate,  and  based  on 
the  enumeration  of  all  the  ways  the  subsystem  probabilities  can  occur  excluding  all  the 
enumerations  which  exceed  the  lower  system  reliability  limits.  It  is  dependent  on  the  following 
assumptions: 

•  The  true  reliability  of  the  total  population  is  based  on  the  reliability  of  the  sample  except 
for  the  zero  failure  case. 

•  The  lower  subsystem  reliability  limit  is  dependent  on  the  confidence  level,  sample  size 
and  reliability  estimator.  Since  this  is  the  lower  subsystem  reliability  limit,  then  the  product  of 
these  lower  limits  gives  the  lower  system  reliability  limit. 

•  The  system  confidence  ievel  is  determined  from  the  allowable  subsystem  probability  of 
occurrence.  In  some  instances,  a  subsystem  probability  of  occurrence  outside  of  the 
confidence  interval  is  used  in  calculating  the  system  confidence  level  due  to  the  reliability 
product  rule. 

The  mechanics  of  this  method  are: 


| 

i  5 
I 


■9  - 

If 

1 

I  i 
I  I 

i  I 
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First,  compute  the  point  reliability  estimate  for  each  subsystem  based  on  the  number  of  test 
runs  and  the  respective  number  of  successes  using 


R(t^)  =  n(t±) 


N. 


or  rather  x,  _  „ 
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where  n(t)  =  x  =  number  of  successes  in  N  trials  or  tests  and  Re  is  the  reliability  estimator. 
Since  the  reliability  point  estimator  obtained  from  the  sample  test  results  of  the  zero  failure 
case  cannot  be  used  as  the  unbiased  reliability  estimator  since  it  is  1.0,  he  used  the  reliability 
estimator 


Rg  N±  -  y±  or  ^  =  a-ei)1/Ni. 


(See  Method  No.  4) 


With  the  respective  reliability  estimator  for  each  subsystem,  he  calculated  the  probability  of 
occurrence  for  each  subsystem  from  the  binomial  formula 


i  =  P(x±  -  D1)  =(x^)re  Xi  (1-Rj,  )  Ni“Xi 


i  i 

where  D,  =  N.-A,  such  that  A,  is  the  number  of  failures  in  N.  tests  all  of  the  ith  enumeration. 


Calculations  were  then  made  in  which  the  probability  of  occurrence  (Gi)  for  each  subsystem 
was  accumulated  until  just  before  it  was  greater  than  the  confidence  value  B„  Then  using  the 
corresponding  point  estimate  reliability  as  the  lower  subsystem  reliability  limit,  he  multiplied 
each  lower  subsystem  reliability  limit  to  calculate  the  lower  system  reliability  limit  such  that 

\s  i  «U>  «L2>  •  •  •  «W  • 

Note  that  only  those  reliability  enumerations  which  did  not  exceed  the  lower  system 
reliability  limits  were  considered.  He  then  summed  all  of  the  acceptable  enumerations, 
governed  by  the  above  equation,  to  give  the  system  confidence  level 


N 

s 

C  =  E  (G  )  . 
S  i=l 


It  should  be  observed  that  as  the  number  of  subsystems,  tests,  and  failures  increase,  ♦he  trail 
of  all  the  possible  combinations  of  probabilities  of  occurrence,  which  are  governed  by  the 
pacing  equation 

\s  -  '  *  *  ^LNS^  * 

will  become  a  monstrous  problem  and  demand  a  great  deal  of  computer  time. 
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6.  1  he  next  method  is  attributable  to  several  authors  who  have  independently  arrived  at  or 
proposed,  similar  methods  of  solution.  An  explanation  of  this  method  will  mainly  be  taken 
from  the  paper  by  Dutoit  [20]  and  one  by  DeCicco[18].  Two  other  references  which  mention 
the  use  of  this  solution  are  Lloyd  and  Lipow  [41]  and  Shooman  [66]. 


This  technique  is  general  in  the  sense  that  it  is  not  limited  to  only  a  series  configured  system 
but  can  have  a  series-parallel  structured  system  in  any  configuration.  The  underlying 
conditions  of  this  approach  are:  (a)  the  components,  subsystems,  and  system  are  described 
by  mathematical  functions  composed  of  binomial  parameters,  (b)  the  “error  propagation” 
formula  is  used  to  define  the  expected  values  as  well  as  the  variance  of  a  function, 
and  (c)  the  Tchebycheff  inequality  is  used  in  order  to  yield  a  one-sided  confidence  limit. 


From  the  derivation  and  results  of  the  “error  propagation”  foimula,  a  function  of  n 
variables  f(Xi,  x2.  .:..,x„)  which  is  expanded  into  a  Taylor  series  about  a  particular  point,  say 


xi,  x2,  ...,  x„,  gives 


f(x^,  X2»  . ..,  x^)  =  f(Xp  &2,  ...»  xr)  + 


,  ~  ,  3f 

<xi  -  *i>  al. 


,  ~  .  af 

v  n  n  3x 


A  A 


^1  *  x2  ’  *  *  *  * 


+  . . .  + 


x^,  •••»  +  higher  order  terms. 


If  the  higher  order  terms  are  neglected  and  the  assumption  is  made  that  the  random 

.  .  .  A  A  A 

variables  are  independent  with  means  at  xk  x: . xn,  the  expected  value  of  the  function  is 

given  by 


E{f (x^,  X2,  ...»  xn>)  -  X2>  *n) 


and  the  approximate  variance  given  by 


3f 


VAR{f(x^,  x2,  x^)}  -  VAR  (x^) 


A  A  A  4 

x2*  *  *  *  *  xnJ 


+  ...  +  VAR(x  ){|^ 
n  dx 


A  A  A  l2 

x2’  ••*»  xn'  • 


[8,  p-  62] 


/ 
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The  equation  for  the  reliability  of  a  component  is 


Ri  =  ML 


where: 

x,  =  the  number  of  successful  functionings  of  the  i,h  item  (component)  under  test 

M,  =  the  total  number  of  tests  of  the  ilh  component 

R,  =  a  best  estimate  of  a  binomial  parameter, 
which  is  a  proportion  describing  a  population  where  the  proportion  R,  of  the  individuals  has  a 
certain  characteristic  at  a  50  per  cent  confidence  level.  The  equation  for  the  reliability  of  a 
system  (series,  parallel,  r-out-of-n,  a  combination,  etc.)  is  written  in  the  usual  way  but  with  the 

A  A 

component  estimates,  as  described  above,  such  that  R=f(Rj).  The  total  variance  of  the  system 
reliability  is  then  calculated  from  this  equation. 


After  determining  the  necessary,  respective  parameters,  the  authors  determined  the 
confidence  interval  which  was  developed  from  Tchebycheffs  inequality  through  the  use  of 
distribution  free  methods.  [8,  p.  34] 

These  results  were  then  utilized  in  the  general  equation  for  computing  the  lower  confidence 
limit  as 

A 

Per  cent  Confidence  Level  R  >  R  -  Aor 

where  “A”  is  not  dependent  on  the  distribution  of  R  and  is  deiived  from  the  inequality. 

When  using  and  developing  this  method,  the  equations  (mathematics)  can  become 
cumbersome  to  work  with,  thus  all  authors  referenced  have  deleted  the  necessary  derivations 
which  explain  and  justify  their  results. 


7.  Lioyd  and  Lipow  [41]  have  covered  the  basics  as  discussed  in  Shooman  [66],  reference 
method  3,  and  have  presented  rather  thorough,  very  excellent  coverage  of  Reliability 
Estimation  and  confidence  limits  in  chapters  7  and  8  (with  appendices)  as  well  as  sections  9.2.2 
and  9.2.3.  An  attempt  to  discuss  their  literature  will  not  be  given  here,  but  attention  is  called  to 
their  work  and  the  derivations  of  exact  system  confidence  limits  which  are  presented. 


'Jti 
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Due  to  the  mathematics  and  involved  nature  of  the  methods,  they  are  not  readily  adaptable 
to  the  aim  of  this  paper.  For  a  brief  but  thorough  discussion  of  their  work  (methods),  see 
Rosenblatt  [61]  and  Schick  [64], 

8.  A  very  excellent  paper  confined  to  the  problem  of  estimating  a  probability  (reliability) 
associated  with  a  system  via  using  data  obtained  from  tests  of  subsystems  or  components  was 
presented  by  Rosenblatt  [61].  This  paper  is  especially  singled  out  as  suggested  reading  if  the 
subject  of  confidence  limits  for  the  reliability  of  complex  systems  and  associated  confidence 
level  is  of  interest.  The  problem  of  confidence  limits  for  system  reliability  is  formulated  in  a 
general  manner  but  with  the  necessary  and  appropriate  mathematical  exercise.  The  literature 
on  the  subject,  up  to  the  publication  date  of  the  paper,  is  more  than  adequately  covered  as  well 
as  explanations,  problem  definitions,  and  comparisons  of  “exact”  methods  with  alternative 
approximate  methods. 

9.  One  of  the  best  known  methods  for  combining  the  component  reliability  estimates  to 
determine  approximate  confidence  limits  for  the  system  reliability  was  developed  by 
Madansky  [42].  A  synopsis  of  the  Madansky  method  can  be  found  in  the  papers  by  Shick  [64], 
Rosenblatt  [61],  Johnson  [34],  and  others.  Iris  based  on  the  observed  failures  of  the  individual 
components  and  assumes  that  the  failures  are  independent  and  that  for  each  component  they 
are  binomially  distributed  with  the  actual  component  reliability  being  unknown.  He  uses  a 
likelihood  ratio  test  based  on  the  fact  that  minus  two  time  ihe  logarithm  of  the  likelihood  ratio 
is  distributed  asymptotically  as  chi-square  with  one  degree  of  freedom,  which  is  discussed  by 
Wilks  [82].  Madansky  then  derives  the  confidence  limits  baf.ed  on  a  general  method  for  testing 
hypothesis. 

This  method  uses  the  likelihood  ratio  test  for  testing  the  null  hpyothesis 


F.  :  f  (R- ,  R0 , 
o  a  2’ 


•  •  *  9 


against  the  alternate  hypothesis 

A  A  A  A 

f(R1,  R2,  Rn)  <  Rg 

A  A  A 

where  f(Ri,  Rj,  ...,  R„)  is  the  system  reliability  structure  function  with  n  subsystems  or 
^  .. 

components  and  R,  =  Xi/n,  is  an  estimate  of  the  i‘  subsystem  reliability,  such  that  Xi  successes 
were  observed  in  n,  trials. 
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R$  is  an  estimate  of  the  system  reliability. 


The  likelihood  ratio  statistic  is  then  given  by 


p  =  L/L 


where 


max 


n 


L  =  max  II  b(x  ,n.,R.) 
i=l 


f  =  R 


and 


0<R.<1 


such  that  b{Xi,n„R,)  is  the  binomial.  (A  brief  review  of  maximum-likelihood  estimators  can  be 
obtained  by  referring  to  Shooman  [66,  p.  87]). 

Thus  the  confidence  interval  for  system  reliability  is  found  as  the  set 

ftsl-2  1ap<£Ytl) 

which  is  the  set  of  ft/s  that  were  not  rejected  as  null  hypotheses  by  the  likelihood  ratio  test 
when  Xj  ^  =Xa:  j  vv*1‘c*1  's  uPPer  100<*  percentile  of  the  chi-square  distribution  with 

one  degree  of  freedom. 

It  should  be  noted  that  this  approximation  is  inapplicable  when  there  are  no  observed 
failures  and  that  it  requires  a  computer  program  for  solution.  It  should  be  used  for  moderate 
reliabilities. 

10.  A  Bayesian  procedure  was  originated  by  Mardo,  Cole,  Seibel,  and  Stephenson 
reference  [74]  which  yields  good  results.  The  procedure  assumes  the  system  prior  reliability  is 
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b''  =  b'  +  O^-S^ 

where  N,  and  S,  are  the  sample  size  and  number  of  successes  in  the  i,h  subsystem.  The  means 
and  variances  of  the  subsystem  posterior  reliability  distributions  are  then  calculated  using  the 
formulas  for  these  parameters  from  the  beta  distribution. 

The  system  posterior  mean  and  variance  is  determined  from  the  subsystem  means  and 
variances  assuming  the  reliability  product  formula  for  a  series  system  and  using  the  above 
formulas  for  the  mean  and  variance  of  a  product  of  independent  random  variables.  The  beta 
parameters  of  the  system  are  then  calculated  algebraically  from  the  mean  and  variance  using 
the  method  of  moments  as  before. 
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i  he  lower  Bayesian  confidence  limit  for  system  reliability  (Ri.)  is  then  calculated  by 
integrating  the  posterior  reliability  density  function  between  this  limit  and  one,  and  setting  the 
integral  equal  *o  the  confidence. 

Whether  or  not  one  agrees  with  the  Bayesian  approach  or  the  assumption  of  a  uniform  prior 
for  the  system  reliability,  the  method  appears  to  yield  good  results  according  to  a  Monte  Carlo 
simulation  study  reported  in  the  referenced  literature. 

II.  Confidence  intervals  for  a  system  of  binomial  parameters  have  been  addressed  by 
several  authors.  A  few  of  the  most  prominent  ones  are  Clopperand  Pearson  [15],  Buehier[IO], 
Brownlee  [9J,  and  Mood  and  Graybill  j48).  A  brief  explanation  of  Buehler’s  method  is 
presented  here. 

In  Buehler’s  article,  he  uses  the  Poisson  approximation  to  the  binomial  distribution  for 
obtaining  one-sided  confidence  limits  for  the  product  of  two  binomial  parameters,  such  that 


is  replaced  by 


-A.  d. 

:  *A.  1 

_ x 


where  n,  is  the  test  sample  size  of  the  iIh  component,  d.  is  the  number  of  failures  observed  in  that 
sample  and  q,  is  the  failure  rate  of  the  i:h  component  so  that  A,  =  n,q,. 

If  q  (di)  and  q  (d:)  arc  the  upper  one-sided  confidence  limits,  with  x/y"  confidence 
coefficient  (level),  for  the  failure  rates  of  two  icspective  components  of  the  system,  whose 
failure  rates  are  qi  and  q:,  then 

Pr  {qx  £  q  (d^)  }  >y/y 


and 


Pr  {q  <  q(dj}  >y/y 


thus 


Pr  *qlq2  —  ')(d1)9(d2^  1  iWj.))  Pr{q2<q(d2)}  _>  y 
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This  method  is  specialized  for  small  probabilities  of  failure  and  for  moderate  sampb  sizes 
which  exceed  40.  The  rationale  for  the  lower  limit  is  developed  the  same  way  as  for  the  upper. 

12.  A  practical  way  of  solving  the  situation  in  which  the  system  reliability  function  is  a 
product  of  dissimilar  functions  is  by  the  use  of  a  Monte  Carlo  simulation.  This  method  has 
been  proposed  by  several  authors,  a  few  are  Orkandj  54],  Levy  [37],  Levy  and  Moore  [38],  and 
Mann  f43]. 

This  method  is  developed  as  follows  for  estimating  the  system  reliability  value  of  a  multi- 
variable  system  function. 

•  Obtain  a  value  for  each  variable  in  the  system  function  by  randomly  sampling  its 
respective  probability  density. 

•  Substitute  this  value  into  the  re'iability  system  function  to  obtain  a  sample  reliability 
value,  system  reliabibty  estimate,  for  a  specified  mission  time. 

•  Repeat  the  above  steps  until  a  sufficient  number  of  system  reliability  point  estimates 
have  been  obtained. 

•  Evaluate  the  point  estimates  of  system  reliability  in  order  to  determine  the  confidence 
limits  for  the  desired  confidence  level.  This  step  is  accomplished  by  ordering  the  system  point 
estimates  in  increasing  magnitude,  thus  building  a  step  wise  cumulative  distribution  of  the 
system  reliability  with  each  step  being  of  height  1  /  n,  where  n  is  the  number  of  system  reliability 
point  estimates  obtained.  System  confidence  limits  are  then  obtained  by  choosing  the  value 
corresponding  to  the  desired  limit  percentage  point. 

The  utility  of  this  method  is  limited  by  its  dependence  on  a  computer,  the  requirement  that 
each  respective  parameter  probability  density  function  be  known  as  well  as  how  weli  it 
describes  the  variations  of  the  parameter,  and  how  many  sample  point  estimates  are  used  to 
describe  the  system. 

13.  The  Bayesian  approach  for  d:termining  sysiem  reliability  with  confidence  has 
received  increased  interest  in  recent  years  jnd  has  created  a  controversy  between  classical  and 
Bayesian  statisticians.  Quite  a  few  authors  have  begun  to  address  this  method  and  the 
controversy  that  has  evolved.  Comparisons  and/or  comments  on  the  Bayesian  versus  the 
classical  are  addressed  by  Easterling  {21],  Crelin  f  17],  Canavos  [13],  Schafer  [63 j.  Bonis  [7], 
and  others.  Application  of  the  Bayesian  approach  is  given  by  Schafer  [63]  and  his  other 


articles  as  discussed  in  this  reference;  Wolf  { 83 1;  Springer  and  Thompson  [69,  70];  Zimmer, 
Breipohl,  and  Prairie  [84|;  Hahn  and  Shapiro  [ 30];  Fertig  [25];  and  others. 


This  method  is  based  on  Bayes’  Theorem  which  follows  directly  from  the  concept  of 
conditional  probability  and  provides  a  mechanism  for  combining  the  initial  or  prior 
probability  concerning  the  occurrence  of  some  event  with  related  experimental  data  to  obtain 
a  revised  or  posterior  probability.  This  theorem  stated  as; 


Pr (B |  A . ) 

Pr  (AjB)  =  Pr  (A±)  _  f 

n 

2  Pr(BjA.)Pr(A.) 


has  a  two  term  right-hand  side:  Pr(A,),  the  prior  probability,  and 


Pr(B|Ai) 


n 

2  Pr(B|A.)Pr(A  ) 
i=l 


the  factor  by  which  the  prior  probability  is  revised  on  the  basis  of  the  experimental  data.  [30,  p. 

21] 

Although  the  use  of  this  method  for  determining  system  reliability  from  component  or 
phase  test  data  seems  to  be  promising,  since  it  is  neither  limited  to  specific  distributions  nor 
test  sample  size,  it  has  not  been  accepted  for  use  in  reliability  assessment  of  weapon  systems 
due  to: 

«  The  difficulty  in  determining  the  prior  distribution. 

•  A  lack  of  feel  for  the  sensitivity  of  a  Bayes  point  estimate  to  the  prior  distribution. 

•  An  inherent  dislike  for  mixing  intuitive  information  with  real  data,  since  it  deteriorates 
the  objectivity  of  the  analysis. 

•  The  method  is  not  really  understood. 
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14.  In  concluding  the  leview  of  the  literature  on  the  subject,  several  additional  references 
and  methods  will  be  mentioned  but  no  attempt  will  be  made  to  give  an  explanation  or 
comment.  This  portion  will  only  serve  to  alert  the  reader  to  what  exists  on  the  subject  and 
where  it  might  be  found.  The  author  hopes  that  the  above  explanations  and  the  rather  cursive 
along-the-scale  view  of  the  methods  that  were  presented  has  given  the  reader  an  introduction 
to  the  variety  of  methods  that  might  be  applied  toward  the  solution  of  the  problem.  The  author 
also  hopes  that  the  reader  has  been  left  with  an  understanding  of  the  applicability  of  the 
methods  such  that  he  can  decipher  which  ones  seem  reasonable  and  which  ones  seem  to  be  in 
“left-field.” 

Articles  addressing  the  reliability  of  multi-component  complex  systems  have  been  written 
by  Abraham  [1];  Messinger  and  Shooman  [46];  Birnbaum,  Esary,and  Saunders  [6];  Tomsky, 
Chow,  and  Schiller  [77J;  Irwin  [33];  Barlow  and  Proschan[4];  Pieruschka[56];andTakenaga 
[76].  Comparisons  of  various  methods  have  been  made  by  Myhre  and  Sanders  [50],  and 
Aggarwal,  Misra,  and  Gupta  [3].  The  explanation  of  confidence  intervals  and  limits  with 
respect  to  component  and  system  reliability  is  given  by:  Neyman  [51];  Ireson  [32];  Bazovsky 
[5];  Roberts  [60];  Mann  [44];  Goodman  and  Madansky  [27];  Soanes  [72];  and  Proschan  [57]. 
Articles  on  computer  programs  for  obtaining  confidence  intervals  have  been  written  by 
Engelrnan,  Roach,  and  Schick  [23],  and  Springer  and  Thompson  [71].  Specific  subjects  of  Pr 
(Y  <  X)  has  been  addressed  by  Moore  and  Taylor  [49];  systems  with  few  failures  by  Saunders 
[62];  dependent  failures  by  Shooman  [67];  bounds  and  propagation  of  uncertainties  in  system 
reliability  by  Shaw  and  Shooman  [65];  reliability  of  series  systems  from  component  test  data 
by  Connor  and  Wells  [16],  and  Mann  and  Grubbs  [45];  and  systems  analysis  by  Whitehoi  se 
[81]. 

2.  AN  ESTIMATE  OF  THE  LOWER  CONFIDENCE  LIMIT  ON 
SYSTEM  RELIABILITY 

This  chapter  uses  some  of  the  previously  presented  concepts  and  definitions  to  create  a 
confidence  limit  at  a  specified  level  on  missile  weapon  reliability  without  having  to  resort  to 
some  rigorous  and  lengthy  solution.  It  establishes  an  estimate  of  the  lowest  value  necessary  in 
order  to  meet  confidence  specifications,  thus  establishing  a  lower  bound  on  the  required 
reliability  interval. 

The  suggested  method  adopts  the  technique  for  generating  a  one-sided  tolerance,  or  control 

•  A 
limit  in  construction  of  statistical  quality  control  charts.  This  limit  is  given  by  the  quantity  R  - 

Kctr  with  the  property  that  the  probability  is  y  that  at  least  a  proportion  (1  -  /3)  of  the 

distribution  will  be  contained  within  the  interval  R.  -  Kctr  and  plus  one;  where  R  is  the  point 


estimate  of  weapon  reliability,  K  is  a  tolerance  factor,  or  is  the  standard  deviation  of  the 
reliability  model,  y  is  the  confidence  level,  and  /3  is  the  level  of  significance.  This  control 
equation  is  applied  in  the  following  manner: 

•  Construct  a  weapon  reliability  model  from  which  the  point  estimate  of  weapon 
reliability,  R,  is  obtained.  Component  estimates  in  the  model  are  obtained  from  observations 
(statistical  data)  taken  during  subsystem  and  system  level  tests. 


•  Determine  the  variance  and  thus,  the  standard  deviation  of  the  model.  This  is  best 
accomplished  by  use  of  the  generation  of  system  moments,  sometimes  referred  to  as  the  “error 
propogation  method”  which  does  not  require  assumptions  concerning  the  form  ot  the 
reliability  phase  distributions  nor  their  test  sample  size.  This  is  primary,  since  the  reliability 
phase  distributions  are  unknown  and  the  phase  test  sample  sizes  are  usually  different. 

•  Determine  the  tolerance  factor,  K.  This  is  done  by  use  of  the  Camp-Meidell  inequality 
which  is  a  distribution-free  and  fairly  conservative  method  and  one  which  applies  to  any  set  of 
numbers  whether  the  numbers  are  viewed  as  constituting  a  sample  or  a  population.  The  most 
general  and  easily  applied  use  of  this  inequality  is  restricted  to  certain  circumstances.  These 
circumstances  are: 


—  that  the  distribution  be  unimodal 

—  that  the  mode  approximately  coincide  with  the  arithmetic  mean 

—  that  the  distribution  be  smooth. 

The  unimodal  case  has  been  used  in  this  paper  since  the  majority  of  the  distributions  that 
would  describe  the  results  of  test  data  on  missile  systems  come  very  close  to  meeting  the 
prescribed  conditions.  Camp  [  1 2]  presents  the  cases  for  c  =  0, 1 ,  and  2,  where  c  is  a  measure  of 
skewness  such  that  the  inequality  is  applied  to  functions  with  more  than  one  mode.  He 
emphasizes  the  case  for  c  =  0  (one  mode)  and  points  out  that  in  a  badly  skewed  function,  the 
origin  might  be  chosen  at  the  mode  instead  of  the  mean.  Under  this  circumstance,  the 
probability  and  moments  could  be  defined  with  respect  to  the  new  origin,  thus  giving  the 
unimodal  case. 

•  Substitute  the  values  obtained  for  the  above  parameters  into  the  control  equation, 

■A 

R-Kc;r,  to  yield  the  lower  limit  on  weapon  reliability  at  the  designed  confidence  level. 
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A.  CONSTRUCTION  OF  THE  RELIABILITY  MODEL  AND  ITS  VARIANCE 

Considering  a  series  configured  system  as  one  which  creates  a  single  path  from  cause  to 
effect,  the  requirement  that  ail  elements  work  successfully  for  system  success  is  imposed.  Thus, 
the  event  representing  system  success  is  the  intersection  of  the  x’s  where  x,  is  the  event 
signifying  the  success  of  the  ilh  element  and  i  =  1, .  .  m  such  that  the  system  is  made  up  of  m 
elements.  In  other  words,  the  probability  of  system  success  is  the  reliability  of  the  system  given 
by: 

R  =  P  =  P(x,x_  . . .  x  ) 
s  11  m 


and  if  the  events  are  independent 


m 

r  =  n  p(x,>.  (i) 

i=l 

For  a  parallel  configured  system,  the  probability  of  success  is  given  by  the  probability  of  the 
union  of  the  m  successful  events,  as: 

R=P  =  P  (x,  +  x„  +  . . .  +  x  ) 
s  1  2.  m 


and  if  the  events  are  independent 


m 

R  =  1  -  II  P(x  ).  (2) 

1=1 


where  Xi  represents  the  failure  of  the  i'h  element. 


Considering  a  general  series  —  parallel  configured  system,  the  reliability  equation  would  be: 


R  =  {l-(l-x1)a}{l-(l-x2)b}...{l-(l-xi)t}...{l-(l-xm)k} 


or 


R  =  n  {l-(l-x  )fc}  (3) 

i=l 
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where  t  elements  arc  in  parallel  in  set  i  in  which  all  x’s  have  the  same  probability  and  there  arc 
m  sets  in  the  system.  This  is  illustrated  in  Figure  I.  (The  r-out-of-n  case  is  presented  in  Chapter  3.) 


Figure  1.  Series  -  parallel  configured  reliability  model. 

After  determining  the  system  reliability  structure  model,  which  would  be  one  or  a 
combination  of  the  above  configurations,  it  is  necessary  to  determine  its  standard  deviation. 

A 

This  value  is  required  as  a  component  in  the  control  equation  R  -  Kor.  In  order  to  determine 
this  value,  the  following  equation  is  used  from  Hahn  and  Shapiro  [30,  pp.  23 1,255]  and  Tukey 
[78,  pp.  V1-VI2]. 


m 

Var(R)  =  E 
i=l 


m 

Var (x.)  +  E 
1  i=l 


E(x1  -  E(x^) 


(4) 


Since  all  values  were  determined  from  tests  on  samples,  either  directly  or  indirectly,  they  are 
estimates  of  the  respective  event  they  represent. 

Since  the  emphasis  has  already  been  placed  on  attribute  values  representing  successful  or 
failed  events,  due  to  test  structuring,  the  discussion  will  be  restricted  to  the  use  of  the  binomial 
distribution.  (It  should  be  noted  that  the  x.’s  (i  =  1  .  .  .  m)  can  generally  be  described  by  any 
distribution  and  not  just  the  binomial.)  Therefore,  applying  equation  (4)  to  determine  the 
variance  of  a  series  configured  system  of  independent  sets  it  becomes: 


Var  (R)  =  E  (P1.P2...Pi_rPi+1...piB) 
i=l 


„  x  2  Pi^i 


where  n,  is  the  recorded  number  of  binomial  tests  on  the  ith  set,  p,  is  the  associated  binomial 
parameter  for  the  set  of  which  p,  denotes  the  observed  proportion  of  successes  in  the  n,  trials 


(the  attribute  test  data  p,  =  S ,/  n,).  q,  is  the  binomial  parameter  associated  with  the  failures  of 
the  i,h  set  (q,  =  1  -  p,),  and  there  arc  m  sets  in  the  system. 

Similarly,  the  variance  of  a  parallel  configured  system  of  independent  elements  can  be 
written  as: 

Var(R)  =  [tU-p)*"  2  £3.  (6) 

L  J  n 

where  there  are  t  identical  elements  in  parallel  comprising  a  set  of  which  p  is  the  binomial 
parameter  tor  the  set,  and  n  is  the  recorded  number  of  binomial  tests  performed  on  the  set.  The 
second  term  an  the  right  hand  side  of  equation  (4)  contributes  only  slightly  in  this  case, 
therefore,  it  was  dropped.  The  slight  contribution  that  it  would  have  made  would  lend  to 
decrease  the  value  given  by  the  first  term  on  the  right  hand  side  of  equation  (4).  Thus,  in  this 
case,  equation  (6)  represents  a  maximum  value  for  system  variance. 

Using  the  same  rationale  for  designating  the  parameters  for  the  system  variance  as  used  in 
the  series  and  parallel  model  cases  and  per  the  illustration,  Figure  1,  the  variance  for  a  series- 
parallel  configured  model  is: 


Var(R)  {  [x-(l-px)  ]  ...  [l-fl-p^)  ][l-(l  Pi+1)  ] 

...  [l-(l-?n)k][t(l-Pi)t-1]  )2  ^  ' 


(7) 


B.  DETERMINATION  OF  THE  TOLERANCE  f .  >CTOR,  K,  AND  LOWER 
SYSTEM  CONFIDENCE  LIMIT  ON  RELIABILITY 

The  Camp-Meidell  inequality  will  be  used  to  determine  the  tolerance  factor,  K,  which  is 
another  term  in  the  control  equation  R  -  Kor. 

This  inequality  is  given  by: 


PKO>  X- 


2.25K 


where  Pk0  is  the  probability  of  confidence  within  the  interval  ±Ko  [68,  p.  97],[55],[  1 2],  If  it  is 
desirable  to  have  90  percent  confidence  that  the  true  system  reliability  will  be  greater  than  or 


equal  to  a  lower  confidence  limit  given  by  R-Kor,  the  value  for  K  is  determined  with  respect  to 
the  specified  confidence.  The  systems  reliability  estimate  and  standard  deviation  are  found  by 
use  of  equations  1,  2  3,  5,  6,  and  7,  as  applicable. 

K.  is  determined  for  a  90  percent  confidence  by: 

.90  >  1 - —y 

2.25K 

such  that 

K  <_  2.108 

and  for  a  desirable  70  percent  confidence,  K  <  1.217.  Thus,  in  the  first  instance  there  is  90 
percent  confidence  that  the  true  system  reliability  is  greater  than  or  equal  to  R  -  2.108or. 

C.  NUMERICAL  EXAMPLE  AND  COMPARISON  OF  METHOD 

Consider  a  series  system  reliability  model  made  up  of  5  subsystems  or  reliability  scoring 
phases,  each  of  which  has  a  true  fraction  defective  of  .01  (q  =  .01)  and  each  of  which  was 
subjected  to  50  binomial  tests.  It  is  required  to  have  90  percent  confidence  in  system  reliability 
and  is  desirable  to  determine  the  lower  confidence  limit  on  the  system  reliability. 

From  equation  1,  the  reliability  of  the  system  is  R  =  .95099.  This  value  is  taken  as  an 
estimate  or  as  the  true  system  reliability  depending  on  whether  or  not  the  assumption  is  made 
that  the  true  reliability  of  the  subsystems  or  phases  is  as  designated.  In  order  to  determine  the 
system  standard  deviation,  equation  5  is  used  which  gives  ctr2  =  .000914  (or  =  .03022)  and  the 
tolerance  factor  is  K.  ^  2. 108.  By  combining  these  values  into  the  control  equation,  the  result 
is1 

90  percent  Lower  Confidence  Limit  on  System  R  =  .951-2.1 1 (.03)  =  .888. 

A  comparison  of  5  methods,  each  of  which  evaluates  the  above  example,  has  been  made. 
The  values  for  all  but  the  first  method  (Method  A)  were  generated  from  a  computer  program 
provided  by  Mr.  Dick  Nutt  and  Mr.  Chester  Hopkins  [53]. 

The  purpose  of  the  program  was  to  verify  the  correctness  of  the  candidate  methods  through 
simulation  in  order  to  provide  a  method(s)  for  solution  of  the  problem.  The  problem  is  the 
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inability  to  place  a  confidence  level  on  missile  weapon  reliability  by  use  of  a  procedure  that  is, 
or  will  be,  agreed  to  Army-wide. 

The  program  is  written  for  a  serial  system  reliability  model  of  which  the  binomial  is  the 
parent  population  for  all  testing  and  scoring  phases.  The  three  candidate  methods  used  in  the 
program  were  considered  to  be  the  most  appropriate  ones  for  application  to  missile  weapon 
systems  and  present  testing  philosophy.  The  fourth  method  incorporated  into  the  program 
provides  a  form  of  check  since  it  is  purely  binomial. 


The  methods  compared  are  briefly  described  below  with  the  comparison  results  provided  in 
Table  /. 

Method  A  is  the  proposed  method  presented  in  this  paper. 

Method  B  was  provided  by  Mr.  Ray  Heathcock,  MICOM,  Product  Assurance  Directorate. 
It  is  a  modified  version  of  referenced  method  7  in  the  literature  search  and  is  basically 
explained  in  Lloyd  and  Lipow  [41,  p.  227].  This  method  is  applicable  to  the  binomial  as  the 
parent  population  and  for  an  independent  serial  system  reliability  model,  providing  its  highest 
efficiency  when  sample  test  sizes  are  equal.  The  point  estimate  for  the  model  is  given  as 


k 

R  =  n 

i=l 


where  N,  is  the  number  of  tests  and  f,  the  number  of  failures  in  the  i,h  phase,  respectively.  The 

A 

quantity  Navn  (1  -  R),  where  Navn  is  the  average  test  sample  size  for  the  system,  is  then 
considered  to  represent  the  number  of  system  failures,  F,  in  Navn  tests  on  the  system.  Using 
these  values,  an  estimate  of  the  lower  confidence  limit  on  system  reliability  is  found  from  the 
binomial  tables. 

Method  C  is  a  generalized  version  of  referenced  method  4  in  the  literature  search.  It  was 
provided  by  Mr.  Chester  Hopkins  and  is  applicable  to  any  combination  of  test  sample  sizes, 
number  of  failures,  or  distributions. 

Method  D  is  the  Bayesian  procedure  which  was  explained  as  method  10  in  the  literature 
search.  It  is  applicable  to  the  binomial  and  uses  the  incomplete  beta  functions  as  the  cognate 
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Method  E  provides  a  lower  bound  on  the  reliability  of  the  system  from  pure  binomial 
computations. 

TABLE  1.  POINT  ESTIMATES  GIVEN  BY  DIFFERENT  METHODS 

(90%  Confidence  (y=0.90),  Assumed  System  Reliability  *  .95099) 


Method 


Point  Estimate  of  the  Lower 
Confidence  Limit  on  System 
Reliability 


From  a  review  of  Table  1 ,  it  is  obvious  that  the  proposed  method  (Method  A)  provides  as 
good  an  estimate  of  the  lower  system  reliability  limit  with  a  specified  confidence  as  any  of  the 
other  candidate  methods.  A  review  of  the  methods  compared  indicates  that  the  proposed 
method  is  as  adaptable  as  any  of  the  other  methods,  and  requires  only  hand  calculations. 

3.  APPLICATION  TO  SNIPER  WEAPON  SYSTEM 

This  chapter  applies  the  proposed  method  to  the  reliability  assessment  in  the  development 
program  of  the  SNIPER  Weapon  System.  This  is  a  fictitious  system  but  is  presented  with 
enough  detail  and  realism  in  order  to  convey  the  involvement  of  testing  and  reliability  scoring 
as  well  as  the  assessment  effort.  The  explanation  is  given  at  the  milestone  decision  point  of  the 
Defense  System  Acquisition  Review  Council  III  (DSARC  III)  between  the  Development  and 
Production  Phases  of  the  System  Life  Cycle,  A  favorable  DSARC  III  decision  authorizes 
production  and  standard  type  classification. 


General  information  is  presented  on  the  deployment  mission  and  system  description.  An 
overall  description  of  the  system  development  and  operational  test  programs,  along  with  the 
testing  philosophy,  is  presented  in  order  to  enhance  a  thorough  understanding  of  the 
reliability  phase  scoring  methodology,  scoring  criteria,  and  phase  definition.  The  application 
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of  the  proposed  method  is  presented  for  the  weapon  and  system  reliability  assessment  of  the 
results  of  the  Prototype  Qualification  Test-Government  (PQT-G)  Test  Program. 

Planning  and  structuring  of  the  development  test  programs,  along  with  reliability  scoring  of 
the  test  results,  were  accomplished  through  the  “jury  of  opinion”  concept.  This  concept  is 
based  on  the  belief  that  a  group  review  is  less  likely  to  overlook  errors  of  judgement  than  is  an 
individual.  It  provides  a  control  on  the  process  through  its  members  (reviewers)  concentrating 
on  identifying  and  reviewing  the  respective  reliability  estimates  rather  than  concentrating  on 
their  detailed  calculations.  The  “jury  of  opinion”  concept  is  accomplished  through  a  SNIPER 
Test  Integration  Working  Group  (STIWG),  comprised  of  the  following  primary  members:  (1) 
SNIPER  Project  Manager’s  Office,  system  developed;  (2)  System  Prime  Contractor;  (3)  US 
Army  Test  and  Evaluation  Command  (TECOM);  (4)  US  Army  Materiel  Systems  Analysis 
Activity  ( AMSAA),  major  systems  evaluator;  (5)  US  Army  Training  and  Doctrine  Command 
(TRADCC),  user;  (6)  US  Army  Operational  Test  and  Evaluation  Agency  (OTEA);  (7)  US 
Army  Logistics  Evaluation  Agency  (LEA);  and,  (8)  Electronics  Research  and  Development 
Command  (ERADCOM),  Office  of  Missile  Electronic  Warfare  (OMEW).  The  reliability 
scoring  committee  is  a  subgroup  of  the  STIWG  and  is  comprised  of  the  majority  of  the 
STIWG  primary  members. 

A.  SYSTEM  DESCRIPTION 

SNIPER  is  a  member  of  the  family  of  Short  Range  Air  Defense  (SHORAD)  weapons 
protecting  the  Field  Army  units.  The  system  will  normally  be  employed  to  provide  low- 
altitude  air  defense  for  battalions,  squadrons,  and  company-size  units  operating  near  the 
forward  edge  of  the  battle  area  (FEBA).  The  system  may  also  be  employed  to  provide  air 
defense  for  surface-to-surface  and  air  defense  missile  sites  and  small  vital  areas  when  no  other 
ground-based  air  defense  means  are  available.  The  weapon  will  also  be  used  in  the  early  phases 
of  airmobile  and  airborne  operations. 


The  SNIPER  Weapon  System  mainly  consists  of  the  Weapon  Round  and  Battery  Coolant 
Unit  (BCU).  The  SNIPER  Weapon  Round  consists  of  a  guided  missile  in  a  launch  tube 
assembly  (the  Missile  Round)  mated  to  a  separable  gripstock.  The  guided  missile  consists  of  a 
guidance  section,  warhead  or  telemetry  section,  the  flight  and  launch  motors,  and  tail 
assembly.  The  separable  gripstock  unit  is  easily  attached  to  a  missile  round  to  provide  an 
operational  weapon  round  and  is  easily  detached  following  missile  launch  for  immediate  use 
with  another  missile  round.  The  weapon  round,  as  defined,  does  include  an  installed  BCU 
which  is  a  one-time  use,  throw-away  item.  It  provides  both  the  electrical  power  and  detector 
coolant  to  the  weapon  prior  to  launch. 
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B.  DEVELOPMENT  AND  OPERATIONAL  TEST  PROGRAM 


Full  Scale  Development  Testing  of  the  SNIPER  Air  Defense  Weapon  System  stressed  the 
coordination  of  test  planning,  combining  of  test  objectives  and  cross-utilization  of  test 
resources  by  all  concerned  test  activities  to  provide  the  most  cost  and  time  effective  program.  This 
was  implemented  through  the  STIWG. 

The  overall  SNIPER  Development  Test  Program  activities  were  Engineering  Design  Test- 
Contractor  (EDT-C),  Engineering  Design  Test-Government  (EDT-G),  Prototype 
Qualification  Test-Contractor  (PQT-C),  Prototype  Qualification  Test-Government  (PQT- 
G),  Operational  Test  II  (OT  II),  and  Production  Prototype  Test  (PPT). 

These  tests  were  designed  to  minimize  the  risk  of  a  major  discrepancy  occurring  during  later 
test  and  evaluation,  to  provide  answers  to  critical  questions  and  issues,  to  support  major 
decision  points  and  key  milestones,  and  to  provide  a  smooth  transition  from  developement  to 
production.  To  obtain  these  objectives,  a  test  philosophy  was  formulated  and  applied  which 
proposed:  ( 1 )  progressive  difficulty  in  flight  tests,  (2)  validation  of  the  computer  simulation  by 
flight  test  data  including  minimum  critical  boundaries  and  low  probability  of  hit  trajectories, 
(3)  the  use  of  simulation  to  define  missile  dynamic  boundaries,  and  (4)  the  use  of  full  scale 
tactical  targets  only  against  trajectories  demanding  maximum  target  performance  such  as 
high  speed  and  difficult  maneuver. 

The  objectives,  under  the  prescribed  test  philosophy,  were  accomplished  by:  ( 1 )  a  controlled 
progressive  build-up  from  piece  part  and  component  testing  through  launch  testing  and  flight 
testing  against  increasingly  more  difficult  targets  in  EDT-C  and  EDT-G,  (2)  a  system  level 
demonstration  in  a  field  environment  in  the  Contractor  Demonstration  (CD)  portion  of  EDT- 
C,  (3)  a  final  system  level  demonstration  in  PQT-G,  and  (4)  a  preproduction  evaluation  during 
PPT. 

C.  RELIABILITY  PHASE  SCORING  CRITERIA  FOR  SYSTEM 
EFFECTIVENESS 

The  SNIPER  scoring  criteria  for  system  effectiveness  were  developed  in  order  to  document 
the  firing  results  in  an  orderly  manner,  and  to  permit  assessment  of  the  system  for  conformity 
to  the  requirements  document.  The  scoring  criteria  contain  the  method  for  the  scoring  of 
SNIPER  flight  tests  from  the  time  the  weapons  are  accepted  by  the  government  through  the 
completion  of  the  mission  or  flight  test. 
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Procedures  were  defined  for  scoring  reliability  and  system  effectiveness.  The  reliability 
scoring  provides  a  performance  measure  of  the  ability  of  the  system  to  operate  without  failure. 
System  effectiveness  is  defined  as  the  probability  that  the  system  will  operate  successfully  (kill 
the  target)  when  called  upon  to  do  so  under  specified  conditions.  The  system  characteristics 
assessed  by  reliability  contribute  to  the  system  effectiveness. 

These  procedures  permit  assessment  of  the  weapon  system  from  stockpile  through  flight 
test.  Each  test  was  scored  as  to  whether  a  valid  test  of  the  system  occurred  or  the  test  was 
invalidated  for  some  specific  reason.  The  valid  tests  were  further  scored  as  to  whether  the  test 
was  successful  or  a  failure  occurred.  The  successful  tests  were  categorized  in  terms  of 
significant  results  and  the  failures  were  identified  as  completely  as  possible  in  order  that 
corrective  actions  could  be  taken  where  practical.  This  scoring  technique  provided  a  method 
for  predicting  future  effectiveness  and  reliability  values  by  recognizing  the  effect  of  corrective 
action  of  certain  failures  that  occurred  in  tests. 


The  system  effectiveness  (Es)  for  a  SNIPER  weapon  against  each  type  of  threat  aircraft  was 
defined  in  the  requirements  document  by  effectiveness  limits.  The  is  an  average  value 
determined  by  testing  or  combination  of  testing  and  simulation  against  a  sampling  of  targets 
presented  within  the  system  performance  boundaries.  The  following  definitions  and  equations 
apply: 


E  =  R  xRcxR-xR,.  xL  X  P«  . 
s  s  pf  f  who  m  det 


E  =  R  x 
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R  x  L  x  ?, 
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where: 


R,  =  Preuse  Reliability:  The  probability  that  a  government  accepted  weapon  and  BCU 
are  capable  of  being  successfully  activated  at  the  point  of  initiation  of  activation  of  the 
weapon.  This  definition  excludes  damage  through  mishandling. 

Rpf  =  Prefire  Reliability.  The  probability  that  a  weapon  and  BCU  will  function  without 
failure  from  the  moment  the  BCU  is  activated  until  the  beginning  of  trigger  pull. 

Ri  =  Firing  Reliability.  The  probability  that  the  weapon  and  BCU  (less  warhead 
detonation  functions)  will  perform  without  any  malfunction  which  would  cause  engagement 
failure  from  the  beginning  of  trigger  pull  to  point  of  closest  approach  to  the  target. 
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R^m  =  Warhead  Detonation  Reliability.  The  probability  that  the  fuze  will  perform 
without  any  malfunction  which  would  cause  mission  failure  from  the  point  of  flight  motor 
ignition  until  the  end  of  flight,  and  that  the  warhead  will  detonate  upon  fuze  command. 

Lm  =  Missile  Lethality.  The  probability  that  a  missile  which  functions  properly  through 
warhead  detonation  will  kill  the  target.  Lm  is  a  function  of  intercept  geometry,  accuracy,  and 
target  structural  characteristics. 

Pd«  =  Probability  of  detection,  evaluation  and  transfer.  The  nrobability  that  a  target 
which  presents  itself  within  the  performance  envelope  of  the  system  will  be  detected, 
evaluated,  and  properly  engaged  by  the  gunner.  For  purposes  of  engineering  development,  the 
performance  of  the  gunner  will  not  degrade  this  probability. 

Rwpn  =  Weapon  Reliability.  The  probability  that  from  battery  coolant  unit  insertion  to 
the  point  of  closest  missile  approach  to  the  target  and  warhead  detonation,  component 
malfunction  will  not  cause  engagement  failure  (R*,*,  =  Rpf  xRrX  R„hd). 

For  the  purpose  of  reliability  testing  to  demonstrate  that  the  system  met  the  specification 
requirements,  a  failure  was  defined  as  any  malfunction  in  the  prefire,  fire,  or  warhead  phase 
which  caused  target  engagement  failure.  During  the  development  program,  the  assessment  of 
the  attained  weapon  reliability  was  based  on  data  from  the  flight  test  program  which  was 
evaluated  in  accordance  with  Figure  2 ,  with  the  following  general  scoring  criteria: 

1.  For  instrumented  flights,  the  initial  criterion  for  scoring  benign  environment,  point 
source  missile  flight  reliability  shall  be  that  the  missile  trajectory’  intersects  a  sphere  centered  at 
the  centroid  of  the  target.  Final  reliability  scoring  will  be  established  by  analysis  of  all 
pertinent  flight  data.  Anomalies  exhibited  in  flight  characteristics,  telemetered  monitor 
functions,  and  target  excursions  at  boundary’  proximities  may  require  a  validated  computer 
simulation  to  determine  final  reliability  scoring. 

2.  The  initial  criterion  for  scoring  piume  target  missile  flight  reliability  shall  be  that  the 
missile  flight  trajectory  intersect  the  space  envelopes  defined  in  Appendix  A.  Trajectories 
which  intersect  the  space  envelopes  will  initially  be  scored  reliable.  Trajectories  which  do  not 
intersect  the  described  space  envelopes  will  initially  be  scored  as  unreliable.  Final  reliability 
scoring  will  be  established  by  analysis  of  all  pertinent  flight  data.  Anomalies  exhibited  in  flight 
characteristics,  telemetered  monitor  functions,  target  excursions,  or  boundary  proximities 
may  require  evaluation  by  a  validated  computer  simulation  to  determine  final  reliability 
scoring.  Missile  trajectories  which  fall  outside  the  envelopes  defined  in  Appendix  A,  evidence 


no  flight  malfunction,  and  the  simulation  reproduces  the  intercept  geometry  within  the 
accuracy  of  the  simulation  validation  plan  shall  be  finally  scored  reliable.  Flight  data  which 
establish  positive  evidence  of  missile  malfunction,  independent  of  flight  intercept  geometry, 
shall  cause  the  flight  to  be  scored  unreliable. 

For  the  pre-use  reliability  phase,  it  was  assumed  that  when  the  weapon  was  issued,  no 
gunner  maintenance  that  would  preclude  the  weapon's  readiness  to  fire  would  occur.  The 
value  for  this  phase  w  ill  be  determined  by  the  Stockpile  Surveillance  Program  during  the  Use 
Phase  of  the  System  Life  Cycle.  However,  during  development  testing,  Rs  was  assessed  from 
the  result  environmental,  nonenvironmental,  and  field  handling  test  results  on  the  missile  and 
weapon  round  prior  to  subjecting  them  to  flight  tests. 

D.  RELIABILITY  PHASE  SCORING  METHODOLOGY 

The  PQT-C  and  PQT-G  test  programs  consisted  of  both  ground  tests  and  flight  tests.  The 
ground  tests  were  composed  of  environmental  tests  to  specification  levels,  while  the  flight  tests 
consisted  of  flight  tests  of  non-conditioned  rounds  (rounds  which  had  not  been  exposed  to  any 
form  of  environmental  conditioning  or  tests)  and  those  rounds  that  completed  the 
environmental  or  field  testing  (conditioned).  In  order  to  optimize  the  generation  of 
engineering  data  from  the  environmental  tests  with  minimum  expenditures  of  resources,  the 
weapon  and  missile  rounds  were  subjected  to  sequential  environmental  levels  that  represented 
a  life  time  of  exposure  to  that  environment.  Some  rounds  were  exposed  to  as  many  as  nine 
sequential  environments  before  being  flight  tested  (see  Appendix  B). 

Data  from  the  PQT-C  and  PQT-G  test  programs  were  used  as  the  reliability  data  base  for 
computing  Preuse  Reliability  Values.  (Preuse  Reliability  is  defined  as  the  probability  that  a 
govei  nment  accepted  weapon  and  BCU  are  capable  of  being  successfully  activated  at  the  point 
of  initiation  of  activation  of  the  weapon.)  AR  702-3,  Army  Materiel  Reliability,  Availability 
and  Maintainability  (RAM),  paragraph  2-12,  Operational  Mode  Summary  and  Missile 
Profile,  indicates  that  “RAM  characteristics  will  be  evaluated  in  accordance  with  the  relative 
frequency  of  uses  defined  in  the  operational  mode  summary,  rather  than  overall  inclusive 
potential  uses  or  at  the  rare  extreme  uses.”  In  other  words,  this  statement  indicates  that 
reliability  should  be  assessed  from  data  representing  all  environmental  levels  and  operations! 
conditions,  not  just  the  extreme  cases. 

With  the  above  in  mind,  it  was  concluded  that  the  SNIPER  Preuse  Reliability  data  were 
comprised  of  two  distinct  samples,  conditioned  and  non-conditioned,  that  must  be  combined 
by  some  realistic  means  into  a  reliability  estimate.  At  one  extreme  are  weapons  only  exposed 
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to  the  transportation  and  handling  necessary  to  ship  from  place  of  manufacture  to  White 
Sands  Missile  Range  (WSMR),  where  they  are  assembled  and  delivered  to  the  flight  range; 
and  at  the  other  extreme  are  rounds  preconditioned  to  life  cycle  levels  of  various  environments 
prior  to  flight  testing.  The  conditioned  rounds  are  subjected  to  sequential  environmental 
conditions  of  up  to  nine  environments.  Close  evaluation  indicated  that  a  single  data  point  fot 
each  conditioned  round  was  unrealistic,  since  the  data  would  represent  a  very  small  segment  of 
the  projected  stockpile/field  condition.  Investigation  of  the  development  of  MiI-Std-8 1 0  life 
cycle  test  requirements  indicated  that  each  dynamic  test  represented  at  least  the  95th  to  99th 
worse  case  condition,  while  the  climatic  tests  were  worse  than  the  99th  percentile.  Assuming 
each  environment  is  meeting  the  conservative  95th  percentile  conditions,  the  following  table 
gives  the  probability  of  any  one  weapon  ever  being  exposed  to  the  specified  level  of  a  single  or 
sequence  of  environments  results. 

TABLE  2.  PROBABILITY  OF  EXPOSURE  TO  A  SEQUENCE  OF 
ENVIRONMENTAL  TESTS 


Number  of  Environmental 
Stresses 

Percent  Exposed  to  Combined 
Environmental  Stress 

1 

5 

2 

.25 

3 

.0125 

4 

.00625 

5 

.00003125 

6 

.0000015625 

7 

.000000078125 

8 

.00000000390625 

9 

.0000000001953125 

As  the  above  data  indicate,  five  percent  of  the  weapons  would  be  exposed  to  the 
environmental  level  of  any  one  environment,  but  only  one  in  ten  thousand  weapons  would 
ever  receive  the  combined  stress  of  three  life  cycle  environments,  and,  of  course,  only  eight 
weapons  in  10  billion  would  ever  be  exposed  to  the  specified  level  of  seven  sequential 
environments.  As  noted,  this  degree  of  sequential  testing  becomes  very  unrealistic  quite 
rapidly. 

In  order  to  make  a  viable  estimate  of  Preuse  Reliability,  the  decision  was  made  that  while 
working  within  the  available  data  base,  the  data  should  be  combined  on  as  equal  a  rating  as 


possible  between  non-conditioned  and  conditioned  rounds.  The  logic  for  this  is  presented  by 
the  following  illustration. 


5TH  PERCENTILE 


95TH  PERCENTILE 


where  r  is  the  reliability  estimate.  This  assumes  that  equal  sample  sizes  will  be  available  for  the 
non-conditioned  rounds  and  conditioned  rounds  (one  datum  point  following  each 
environment). 

As  noted  by  review  of  the  reliability  data  base  for  PQT-G  and  OT  II  ( Figure  3),  the 

assumption  of  equal  samples  for  both  conditioned  and  non-conditioned  rounds  did  not  hold. 

Instead  of  making  r  =  (n  +  r2)/  2,  where  n  and  r2  are  the  means  of  the  5th  and  95th  percentile,  it 
„  n 

was  decided  to  let  r  =  S  r,/  n  and  let  the  reliability  estimate  become  more  pessimistic  with  the 
i=I 

heavier  weighting  of  conditioned  data.  In  addition,  as  review  of  the  data  will  confirm  (see 
Appendix  B),  some  data  points  were  actually  accumulated  after  multiple  environmental 
exposure  instead  of  after  each  exposure,  which  added  an  additional  degree  of  pessimism. 

In  the  final  assessment,  preuse  reliability  data  were  accumulated  from  each  operational 
check  of  the  hardware  by  TECOM  that  was  either  the  first  performed  on  the  hardware  or  the 
first  performed  after  the  hardware  was  exposed  to  a  new  environment.  Each  datum  point  was 
given  an  equal  weight  and  Preuse  Reliability  was  calculated  as  a  success  ratio  (successes/ total 
tests)  for  the  missile  round,  BCU  and  separable  gripstock. 


The  SNIPER  Rs  N  lodel,  Figure  4,  illustrates  the  relationship  of  the  various  weapon  system 
components  and  their  effect  on  R,  or  Preuse  Reliability.  The  model  is  factored  into  two 
distinct  parts,  stockpile  storage  environment  and  field  environment.  The  tests  conducted 
during  PQT-C  and  PQT-G  yielded  data  that  were  primarily  related  to  the  field  environment 
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Figure  4.  SNIPER  Rs  model  for  Es  (Preuse  Phase  —  Basic  Load). 


with  very  little  inference  to  the  stockpile  storage  environment.  For  purposes  of  assessment 
during  ED.  the  .stockpile  storage  factor  was  assigned  a  value  of  1.0.  In  calculating  the  field 
environmental  factor,  it  is  necessary  to  treat  the  separable  gripstock  as  a  redundant  system 
when  considering  the  mission  effects  of  visually  detectable  failure  modes.  The  equations  given 
in  Figure  4  pi  ovide  the  relationship  of  visual  and  internal  (electrical)  failure  of  the  gripstock  to 
mission  success  for  a  basic  load  of  six  missile  rounds  and  four  gripstocks.  No  distinction  is 
made  between  visual  and  internal  failure  of  the  missile  round  and  BCU. 

Appendix  B  gives  a  detailed  summary  of  the  reliability  scoring  of  the  SNIPER  PQT-G 
environmental  testing  from  which  the  scoring  methodology  for  the  calculation  of  Rs  can  be 
followed.  It  provides  the  environmental  test  matrix,  the  reliability  scoring  of  the  test  matrix 
rounds,  a  description  of  specific  problems  found,  and  specifies  the  environmental  levels  used 
during  the  test. 

The  flight  test  results  were  scored  in  accordance  with  the  criteria  by  evaluating  the  required 
input  data  shown  in  Table  3.  The  rationale  for  the  assessment  of  the  Preuse  Reliability  Scoring 
Phase  has  been  given  above.  The  scoring  of  Weapon  Reliability  was  subdivided  into  Prefire, 
Firing,  and  Warhead  Reliability  Scoring  Phases  and  subsequently  divided  for  data  gathering 
and  functional  purposes  as  shown  in  Figure  5. 


A  detailed  summary  of  the  reliability  scoring  of  the  flight  tests  has  not  been  provided.  The 
author  feels  that  Appendix  B  provides  sufficient  answers  to  the  “How”  and  “Why”  of  the 
reliability  scoring  when  supplemented  with  the  following  “When.” 

1 .  Rs  was  scored  for  the  gripstock  and  missile  round  after  each  environment-ground  test. 

2.  Rs  for  missile  round  was  scored  for  all  missile  rounds. 

3.  Rs  for  BCU  was  scored  for  all  BCU’s. 

4.  RPr  was  scored  for  the  gripstock  and  missile  round  after  each  activation,  flight,  and 
ground  test. 

5.  Rf  was  scored  for  the  gripstock  and  missile  round  during  the  flight  test  only. 

6.  Rfuzc  was  scored  for  the  fuze  during  flight  test  only. 

7.  Rw/h  was  scored  for  the  warhead  (w/h)  during  flight  test  only. 
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TABLE  3.  RELIABILITY  SCORING  INPUTS 

ENVIRONMENTAL  DATA  -  BCU,  MISSILE  ROUND,  GRIPSTOCK 
TARGET  PERFORMANCE/POSITION  DATA 
GUNNER  PERFORMANCE 
BCU  PERFORMANCE 

LAND  LINE  DATA  (PREFIRE  -  SPIN-UP,  COOLDOWN,  ACQUISITION  SIGNAL) 
LAUNCH  CAMERA  DATA  -  TIP-OFF,  ROLL  RATE,  GUNNER 
MISS  DISTANCE  -  VISUAL,  MDI,  OPTICS 

RELIABILITY  CYLINDER  -  QUICK  LOOK 
SIMULATION  DISPERSION 
TM  RECORDS  -  GUIDANCE  AND  FUZE  FUNCTIONS 
END  GAME  CAMERAS  -  TOWER  SHOTS 

FUZE  DELAY  TIME  -  W/H  PENETRATION 
W/H  PERFORMANCE 
VISUAL  OBSERVATION 

W/H  PERFORMANCE  -  HIGH  ORDER 
W/H  SELF-DESTRUCT 
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f .  CONFIDENCE  IN  WEAPON  AND  SYSTEM  RELIABILITY 


Army  Regulation  702-3.  Army  Materiel  Reliability,  Availability,  and  Maintainability 
(RAM)  requires  that  DT  li/OT  !i  be  designed  to  evaluate  RAM  characteristics  and  assure 
program  continuation  if  sufficient  reliability  growth  of  the  system  has  been  achieved.  It 
specifies  that  sufficient  reliability  growth  will  have  been  achieved  if  the  minimum  acceptable 
value  (MAV)  is  demonstrated  at  high  confidence  during  the  DT/OT  preceding  the  decision  to 
type  classify  standard.  Thus,  the  regulation  requires  the  evaluation  of  a  lower  confidence  limit 
on  weapon  reliability  at  a  required  (system  specified)  confidence. 

The  lower  confidence  limit  on  weapon  reliability  will  be  evaluated  for  the  SNIPER  System 
by  applying  the  proposed  method  as  described  in  Chapter  2.  The  weapon  reliability  scoring 
breakdown  is  given  in  Figure  5  and  the  pertinent  data  from  the  reliability  phase  scoring  results 
of  PQT-G  are  provided  in  Table  4.  The  sample  sizes  and  percent  defective  for  the  respective 
components  of  each  scoring  phase  provide  a  feel  for  the  wide  range  of  sample  sizes  and  data 
fluctuations. 

For  the  assessment  of  the  lower  confidence  limit  on  Weapon  Reliability,  the  value  for  each 

A 

parameter  in  the  control  equation,  Rwpn  -  K.ORwPn,  is  determined  as  follows: 

A 

1 .  The  value  for  Rwpn  is  obtained  by  use  of  equation  1  from  Chapter  2  and  is  the  product  of 

A 

the  values  for  the  reliability  scoring  phases  given  in  Figure 5.  This  product  yields  Rwp„=  .8592. 

2.  The  tolerance  factor  for  both  a  90  percent  and  70  percent  confidence  value  was  generat¬ 
ed  in  Chapter  2  such  that  K.  <1.2.108  for  the  90  percent  case  and  K  <L  1.217  for  the  70  percent 
confidence  case. 

3.  The  standard  deviation  for  weapon  reliability  is  determined  by  application  of  equation 
5  from  Chapter  2  and  the  substitution  of  the  respective  data  values  from  Table  4.  This  yields 
Var  (Rupn)  =  .00149  and  OR»Pn  =  .0386. 

Substituting  the  above  values  into  the  control  equation,  the  following  is  obtained: 

90  percent  Lower  Confidence  Limit  on  Weapon  R  =  .8592  -  2.108  (.0386) 

90  percent  Lower  Confidence  Limit  on  Weapon  R  =  .778 

70  percent  Lower  Confidence  Limit  on  Weapon  R  =  .8592  -  1.217  (.0386) 

70  percent  Lower  Confidence  Limit  on  Weapon  R  =  .8122 
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TABLE  4.  RELIABILITY  PHASE  SCORING  RESULTS-PQT-G  DATA 


Phase 

Phase 

Element 

Sample 

Size 

Percent 
Defective  (q) 

Preuse  (R  ) 
s 

— 

" 

Stockpile 

Field  Environment 

(assessed  as  1.0  during  ED) 
Gripstock  (Rgf) 
gv  (visual) 

71 

5.6 

gi  (internal) 

67 

3 

Missile  Round  (R  ) 

BCU  (Rbf  )  “ 

153 

10.4 

138 

1.4 

Weapon 

Prefire  (R  ) 
pf 

Gripstock  (R  c  .  .) 

*  pf/gpst' 

493 

3 

Missile  Round  (R  , ,  ) 

pf/m.r. 

309 

2 

BC0  <Rpf/bcu) 

107 

2 

Firing  (Rf ) 

Gripstock  (Rf/gpst) 

201 

2 

Missile  Round  r  ) 

30 

3 

BCD  «f/tc»> 

128 

.8 

Warhead 

Fuze  (Rp  ) 

44 

2.2 

Detonation  (Rwhr)) 

Warhead  (R^) 

6 

0 
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Although  the  application  of  confidence  to  system  reliability  is  not  a  requirement,  it  is  easily 
obtained  by  applying  the  proposed  method.  The  system  reliability  assessment  will  be  obtained 

A  A  A 

for  the  model  R  =  R,  •  R»pn  by  combining  the  model  given  in  Figure  4  and  the  weapon 
reliability  expression  given  in  Figure  5  for  which  the  expanded  equation  is  given  in  Table 5. 
The  necessary  data  are  provided  in  Table  4. 


By  substituting  the  assessed  value  of  each  term  from  Table  4  into  the  expanded  equation  for 

A 

system  reliability  given  in  Table  5,  the  point  estimate  for  system  reliability  is  R  =  .7358. 


Applying  equation  4  from  Chapter  2  to  the  expanded  equation  for  system  reliability  given  in 
Table  5  and  noting  that  E  {x,  -  E(Xi)}3  in  equation  4  is  the  third  central  moment  for  each 
respective  term  in  the  system  reliability  expression  ( Table  5),  the  system  variance  is  calculated 
as  shown  below.  The  calculations  are  shown  for  each  individual  term  in  the  expanded  system 
reliability  equation.  The  values  for  these  individual  terms  are  then  summed,  per  equation  4 
from  Chapter  2.  to  yield  the  system  variance.  The  respective  calculations  are: 

1 .  The  calculations  for  the  contribution  to  system  variance  from  the  first  term,  R^,  in  the 
system  reliability  model  are: 


Var  (R(lst  tern  only.R^)) 


p  A  A«  A-  “j  A 

=  {  12R  -  24R  +  12R  •  R 

L  gv  gv  gvj  g 


A  A  A  a  P  Q 

R  .  *  .  R  -SVi 

mf  %f  wpn  n 

gv 


•;K 


-  24R2  +  12R3  1  •  R  ,  •  R  ,  •  R,  , 
gv  gvj  gi  mf  7>f 


'  <[12- 


A  A^  -|  A 

48R  +  36R  ■  R 

gv  gv  J  gi 


AAA 


R.  -  •  R  )  •  (R  (R  -1) 
of  wpn  gv  gv 


-  (R  -2}+  3R  (R  -1)  +  R  -3(R  ) 

gv  gv  gv  gv  gv' 

■  (R  -1)  -  3R2  +  2R3  } 
gv  gv  gv 

=  {.00068  •  .00074}  +  {.02615  •  (-.90645) 

-  (-.29942)} 


I  ',i^:  . . . .  ■:.:.."~ii.i',:i:.'-;' V,  ..yi  . . .  . . . . . <»n*»«m»H>ll.n»flaSHitj*: 


'' '  1  w"' 


=  .000001  +  .007098. 


Var  (R(lst  term  only,  R  ))  =  .00710. 


2.  The  calculations  for  the  contribution  to  system  varia  "'om  the  second  term,  Rt„  in 
the  system  reliability  model  are: 


Var (R(2nd  term  only,  R  .))  =  {!"  R4  +  4R3  (1-R  )  +  6R2  (1-R  )2] 

gi  L  gv  gv  gv'  gv  gv  J 


2  pgiqgi 


R,  ,  •  R  }  — & 

Df  wpn  n 


{[.99933  ]  •  .8960  •  .9860  •  .85916}2 


.00043. 


Var(R(2nd  term  only,  R^))  =  -00025. 


3.  The  calculations  for  the  contribution  to  system  variance  from  the  third  term,  Rmt,  in  the 
system  reliability  model  are: 

Var (R( 3rd  term  only,  R  ,))  *  {  T {R4  +  4R3  (1-R  )  +  6R2  •  (1-R  )2} 

3  ’  rnf  L  gv  gv  gv'  gv  gv 

:  n  :  :  ,2  pmf  a-mf 

•  R  .  •  R,  c  •  R  }  - 

gi  j  of  wpn  n  c 

mr 

=  {  [{.99933}  •  .9700 j  •  .9860  •  .85916}2 

•  .00061. 

/\ 

Var(R(3rd  term  only,  R^))  =  .00041. 

A 

4.  The  fourth  term,  Rbr,  contribution  is: 


Var(R(4th  term  only,  R^))  = 


{  r {R4  +  4R3  (1-R  )  +  6R2  (1-R  )2} 

L  gv  gv  gv  gv  gv 

•  R  .  1  *  R  •  R  }2 

gi  J  mf  wpn  it  , 


=  {  £{.99933}  *  .9700  ]  •  .8960 


-85916}2 


.0001  . 


Var(R(4th  term  only,  R^))  =  -00006. 


5.  Tiie  fifth  term.  Rr.f  contribution  is: 


Var(R(5th  term  only,  Rpf/gpSt^ 


Rwhd^ 


Rpf/bcu  Rpf/m.r.  Rf 
2  ppf/epstqpf/epst 


pf/gpst 


=  {.8564  *  .9800  •  .9800  -  .9430 
-  .9780}2  *  .000059. 


Var(R(5th  term  only,  R  „))  =  .00003 

pt/gpa,. 


6.  The  sixth  term,  Rpf/bcU,  contribution  is: 


Var(R{6th  term  only,  V/bc„»  =  {8S  8 


s  Rpf/gpst  Rpf/m.r.  Rf 


R  * 
whd 


2  Ppf/bcuqpf/bcu 
npf/bcu 


=  {.8564  •  .9700  •  .9800  -  .9430 


•  .9780}  •  .000183. 


Var(R(6th  term  only,  Rpf/j,cu^  = 


7.  The  seventh  term.  RPf/n,.r.,  contribution  is: 


A  A 


Var (R(7th  term  only,  Rpf/m>r<))  »  {Rs  *  Rpf/gpst  '  Rpf/bcu  *  Rf 


r  ppf /m.r .  pf/m.r . 

whd  npf /m.r. 


=  {.8564  •  .9700  •  .9800  •  .9430 
•  .9780}2*  .00097. 


Var(R(7th  term  only,  r  ))  =  .00055. 

8.  The  eighth  term,  Rr/gPsi,  contribution  is: 


Vai(R(8th  term  only,  Rf/gps{.)) 


{Rs  ’  Rpf  *  Rf/bcu  *  Rf/m.r. 


.  ^  }2  f/gpst4f/gp3t 

Whd  nf/gpst 


{.8564  •  .93159  •  .9920  •  .9700 
•  .9780}2  •  .000098. 


Var (R(8th  term  only,  Rf/gpst))  = 


=  .00006. 


9.  The  ninth  term,  Rf/bcu,  contribution  is: 


A  A 


Var (R (9th  term  only,  Ef/b=u»  “  lR.  •  •  R 


s  pf  f/gpst  f/m.r. 


R 

whd 


2  Pf/bcuqf/bcu 
nf /bcu 


{.8564  • 


93159  •  .9800  •  .9700 


.9780r  •  .000062. 


A 

Var(R(9th  term  only,  Rfybcu))  =  .00003. 


10.  The  tenth  term.  contribution  is: 


/v 

Var (R(10th  term  only, 


lRs  *  Rpf  ’  Rf/gpst  ’  Rf/bcu 
,2  Rf /m.r.^f/m.r. 


R 

whd 


nf /m.r. 


=  {.8564  •  .93159  •  .9800  *  .9920 


.9780}  •  .00097. 


A 

Var (R(10th  term  only,  Rf/m>r.)'  =  • 00056  * 


The  eleventh  term,  R(u«,  contribution  is: 


-  ,A  ~  ~  , 

Var (R(llth  term  only,  Rfuze)>  =  1RS  *  Rpf  *  Rf  '  w/lt 


s  pf 


i  ^fuze  Hfuze 


n 


fuze 


{.8564  •  .93159  *  .9430  *  1.0}^ 
•  .000489. 


Var(R(llth  term  only,  Rfu2e))  =  • 00028 
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I  3 
=  v 


IB 


iss 

ps 

m 


A  1 

12.  The  twelfth  term.  R*  h,  contribution  is: 

A  A  A  A  A  A  n 

Var(R(12th  term  only,  •  (Rs  •  Rpf  •  Rf  •  Rfuza»  j 

*Wh^w/h  i 

nw/h 

=  { .8564  •  .93159  •  .9430  •  .9780}2  I 

*  0  . 

A  A  C, 

Var(R(12th  term  only,  Rwy^))  =0.  i 

Summing  the  calculated  values  (rounded  off  to  the  5th  decimal  place)  from  1  through  12  | 

above,  the  system  variance  is:  « 


Var(R)  =  .00710  +  .00025  +  .00041  +  .00006  +  .00003  +  .00010 
+  ,00055  +  .00006  +  .00003  +  .00056  +  .00028  +  0 
Var(R)  =  .00943. 

Therefore,  <tr  =  .097 1 . 

Combining  the  values  for  all  respective  terms  in  the  control  equation,  the  lower  confidence 
limit  on  system  reliability  is  determined  as: 

R-Ko2  =  .7358  -  2.108  (.0971)  =  .531. 

K 


Therefore,  the 


90  percent  Lower  Confidence  Limit  on  System  R  =  .531 


and  the 


| 


70  percent  Lower  Confidence  Limit  on  System  R  =  .618. 
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5.  CONCLUSIONS  AND  RECOMMENDATIONS 


The  aim  of  this  papet  has  been  met  since  a  method  has  been  presented,  as  described  in 
Chapter  2,  which  prosides  a  quick  and  very  good  means  of  determining  a  definite  lower 
confidence  limit  on  system  reliability  given  a  designated  confidence  level. 

Only  desk  calculations  are  necessary  for  the  application  of  the  method.  It  can  provide  a 
fairly  accurate  reliability  estimate  in  time  to  support  management  decisions  when  not  enough 
time  is  available  for  rigorous,  exacting  analytic  solutions  or  computer  facilities  with  “canned” 
applicable  programs  are  not  available.  It  is  not  limited,  in  that  it  applies  to  practically  ary 
reliability  system  model,  phase  sample  sizes  and  combinations  of  different  phase  distributions. 
It  is  especially  adaptable  and  easily  used  with  the  common  serial  reliability  model  applicable  to 
missile  weapon  systems  and  the  present  testing  philosophy  and  reliability  phase  scoring 
methodology. 

The  recommendation  is  twofold:  (1)  That  the  method  be  used  by  MICOM  and  the 
respective  industrial  community  as  a  “quick-look”  procedure  for  generating  a  lower 
confidence  limit  on  weapon  reliability  during  the  system  development  phase;  and  (2)  that  the 
method  be  used  during  the  production  phase  to  provide  an  “on-the-spot”  check  of  system 
reliability  and  the  inherent  confidence  in  the  production  process. 
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APPENDIX  A 


SNIPER 

DETERMINATION  OF  FLIGHT  RELIABILITY 


To  determine  the  flight  reliability  ( Ri>  of  SNIPER,  various  limitations  for  different  target 
conditions  were  considered.  These  conditions  are: 

a.  Benign,  with  or  without  maneuvers. 

b.  Afterburner,  with  or  without  maneuvers. 

c.  Afterburner  maneuvers,  and  countermeasures. 

Figures  A-I ,  A-2 ,  and  A-3  show  the  space  envelopes  for  the  above  conditions.  The  space 
envelopes  are  cylinders  of  lengths  and  diameters  to  represent  success/ failure  under  different 
target  conditions.  Where  miss  distance  instrumentation  was  used,  a  first  “look” determination 
of  the  success/ failure  was  made.  If  a  flight  was  outside  of  the  specific  space  envelope,  a  valid 
simulation  was  used  to  determine  the  success /failure  condition  by  considering  the  actual  flight 
of  a  typical  good  missile.  In  the  cases  of  gross  failures,  ballistic,  loss  of  acquisition,  or  excessive 
miss  distance  no  simulation  was  required.  In  service  practice  and  gunner  training  firings 
where  only  a  miss  distance  indicator  is  used,  a  first  “look”  determination  will  be  made  and  then 
validated  by  a  valid  simulation  representing  range  conditions  similar  to  the  particular  flight. 


57 


as,; 


■<.:.V.:-*.i  siarg.y  :>■»• 


g< 

i  Z  /*  <J 

cc<2< 

is«si 

Sigoi 

M  S  i 

X  ^  Ul  z 

1 2  2;  <o 

<2s~p 

jyi-So 

ODCCZ 

503- 

<I«D 

0cfe2<_ 

O  ^  U|  CC 

92&o 

UJ  Z  £  <  oc 

o.<5iir 

Oi|couj 

|s°25 

“'ll  si0 
w  uj  o  9  w 

OZtzC5 

<epliz 

WCQ^OCE 


Sui 

OOuj 

ZZj 

g£3 

qP=! 

u.°< 

Si2 


H  1U  O 
3IC 
0£< 
P  uj  £ 


ZcqU 

C3<5 

li.  U)  (£ 

_,  oc  £ 
:z“ 
o? 


igure  A-2.  Reliability  envelope. 


APFENDIX  B 


This  Appendix  summari/es  the  reliability  scoring  of  the  SNIPER  PQT-G  environmental 
testing.  It  provides  the  environmental  test  matrix,  the  reliability  scoring  of  the  test  matrix 
rounds,  a  description  ol  specific  problems  found,  and  specifies  the  environmental  levels  used 
during  the  test. 

Legend:  TC-?  —  Telemeter  round  cold  condition 
TH-?  —  Telemeter  round  hot  condition 
WC-?  —  Warhead  round  cold  condition 
WH-?  —  Warhead  >ounu  hot  condition 
MR  —  Missile  Round 
WR  —  Weapon  Round 
TLM  —  Telemeter 
WH  —  Warhead 
S  —  Success 
F  —  Failure 
NT  —  No  Test 

T.V.  —  Transportation  vibration  test 
36  in.  —  36  inch  drop  test 
L.C.  —  Loose  cargo  bounce  test 
21  in.  —  21  inch  drop  test 
Humid  —  Humidity  test 
S.  Fog  —  Salt  Fog  Test 
Immer  —  Immersion  Test 

WC-1 

Missile  kv/>nd  was  subjected  to  x-ray  inspection  by  the  contractor  and  a  visual  inspection 
prior  to  initiation  of  enviionmentel  tests.  Round  was  subjected  to  transportation  vibration, 
36"  drop,  loose  cargo  and  2 1 "  drop  before  first  operational  check,  i.e.,  polecheck.  Round  was 
incorn  ctly  dropped  on  the  forward  end  during  the  21 "  drop  test  and  the  iR  window  cracked; 
this  was  not  scored  as  a  reliability  failure  due  to  the  incorrect  drop.  The  window1  was  replaced 
by  the  contractor  and  the  round  subjected  to  icing  and  mud  environments.  Visual  examination 
and  x-ray  inspection  were  performed  after  each  environment  with  exception  of  mud  where 
only  a  visual  was  conducted.  Prior  to  flight  test,  the  round  was  subjected  to  operational  test 
(polecheck).  WC-1  successfully  completed  all  environmental  testing  without  failure.  Round 
was  fired  on  20  November  76  as  PQTG-10. 
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TABLE  B-3.  (CONCLUDED) 


WC-2 


I  V,\-* 


Missile  round  was  subjected  to  an  x-ray  inspection  by  the  contractor  and  a  visual  inspection 
prior  to  initiation  of  tests.  Round  was  exposed  to  transportation  vibration,  36"drop,  loose 
cargo  and21"drop  before  initial  operational  check  (polecheck).  Visual  examination  and  x-ray 
inspection  were  performed  after  each  environment.  Round  was  incorrectly  dropped  on 
forward  end  during  21  "drop  test,  resulting  in  internal  damage  to  the  guidance  section.  The 
round  was  returned  to  the  contractor  for  repair  where  the  head  coil  and  detector  were 
replaced.  This  was  not  scored  as  a  reliability  failure  because  of  the  incorrect  drop.  The  round 
was  returned  to  the  test  program  and  subsequently  exposed  to  mud  and  icing  tests.  The  round 
completed  the  environmental  exposures  in  an  operational  condition,  (without  failure)  but  was 
not  flight  tested  because  of  problems  with  target  availability.  The  planned  flight  objective  was 
deferred  until  the  PPT  Program. 


WC-3 


Weapon  Round  received  an  x-ray  inspection  by  the  contractor  and  a  visual  inspection  prior 
to  initiation  of  environmental  tests.  Round  was  subjected  to  transportation  vibration,  36" 
drop,  loose  cargo,  2 1 "  drop  and  icing  before  being  subjected  to  first  operational  check.  Round 
was  then  subjected  to  mud.  Visual  examination  and  x-ray  inspection  were  performed  after 
each  environmental  treatment  with  exception  of  mud  where  only  a  visual  was  conducted.  An 
operational  test  was  made  after  the  completion  of  all  environments,  prior  to  flight  test.  The 
round  exhibited  no  failure  during  environmental  test  sequence  and  was  flight  tested  as  PQTG- 
1.  Round  impacted  ground  due  to  target  presentation  being  too  low;  target  beyond 
boundary  conditions. 

WC-4 

Weapon  Round  received  an  x-ray  inspection  by  the  contractor  and  a  visual  inspection  prior 
to  initiation  of  environmental  tests.  Round  was  subjected  to  transportation  vibration,  36" 
drop,  loose  cargo,  21  "drop  and  mud  before  first  operational  test  (polecheck).  After  this  test, 
round  was  subjected  to  icing.  Visual  examination  and  x-ray  inspection  were  performed  after 
each  environment,  with  exception  of  icing,  where  only  a  visual  was  conducted.  Operational 
test  was  performed  on  the  round  after  completion  of  all  environmental  treatments  and  prior  to 
flight  test.  The  round  had  no  failures  during  the  complete  test  sequence  and  was  flight  tested  as 
PQTG-2. 


WH-I 


Missile  Round  received  an  x-ray  inspection  by  the  contractor  and  a  visual  inspection  prior 
to  initiation  of  environmental  tests.  Round  was  subjected  to  transportation  vibration,  36" 
drop,  loose  cargo  and  21  "drop  prior  to  the  first  operational  test.  During  21  "drop,  the  round 
was  incorrectly  dropped  on  the  forward  end  and  sustained  damage  to  the  sight  mount  and 
sight.  These  items  were  replaced  by  the  contractor  and  the  round  returned  to  test.  No 
reliability  failure  was  charged  because  the  damage  occurred  as  a  result  of  over-test 
environment.  The  round  was  then  subjected  to  rain,  humidity,  salt  fog  and  immersion  with 
operational  tests  following  each  environment.  Visual  examination  followed  each 
environment,  with  x-ray  inspection  conducted  after  each  dynamic  environment  and  prior  to 
flight  test.  After  the  immersion  environment,  the  round  was  found  to  have  a  gyro  caging 
problem.  This  hardware  was  returned  to  the  contractor  for  analysis. 

WH-2 

Missile  Round  received  an  x-ray  inspection  by  the  contractor  and  a  visual  inspection  prior 
to  initiation  of  environmental  test.  Round  was  exposed  to  transportation  vibration  and  36" 
drop.  A  visual  examination  and  x-ray  inspection  was  performed  after  each  environment.  The 
first  operational  test  was  performed  after  36"  drop.  During  the  36"  drop,  the  round  slid 
forward  in  the  launch  tube  and  broke  the  1R  window.  Analysis  indicated  that  the  lock  spring 
retention  force  on  the  umbilical  plug  was  approximately  10  lbs,  which  is  below  specification 
requirements  of  15-25  lbs.  A  deformed  spring  was  the  cause.  No  design  change  is  planned, 
rather  it  is  felt  that  increased  production  control  (x-ray  inspection)  will  resolve  the  problem. 
This  unit  was  not  repaired. 

WH-3 

Weapon  Round  received  an  x-ray  inspection  by  the  contractor  and  a  visual  examination 
prior  to  initiation  of  environmental  tests.  Round  was  exposed  to  transportation  vibration,  36" 
drop,  loose  cargo,  21"  drop  and  rain  before  the  first  operational  test.  Then  the  unit  received 
humidity,  salt  fog,  mud  and  immersion  with  operational  checks  after  each  environment. 
Visual  examinations  were  made  after  each  environment  and  x-ray  inspection  made  after  each 
dynamic  environment.  The  round  failed  the  operational  test  following  the  immersion 
environment  due  to  no  gas  flow;  investigation  revealed  the  unit  had  been  overtested  in 
immersion  by  being  submerged  for  two  hours  instead  of  required  five  minutes.  Analysis  of 
hardware  revealed  a  clogged  gas  line  caused  by  leaking  rubber  wiper  in  gas  line  insert 
assembly.  The  wiper  allowed  water  to  enter  the  line  during  the  immersion  test.  It  is  postulated 
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that  damage  to  the  wiper  was  caused  by  the  polecheck  adapter  since  tests  have  indicated  that 
BCU  insertion  will  not  cause  problem. 

WH-4 

Weapon  Round  received  x-ray  inspection  by  the  contractor  and  visual  inspection  prior  to 
start  of  environmental  tests.  Round  was  subjected  to  transportation  vibration,  36"drop,  loose 
cargo,  21"  drop  and  rain  before  the  first  operational  test  was  made.  After  these  tests  the  round 
received  salt  fog,  humidity  and  immersion  with  operational  test  after  each  environment. 
Visual  examinations  were  made  after  each  environment  and  x-ray  inspection  made  after  each 
dynamic  environment  and  prior  to  flight.  The  round  failed  the  operational  test  following 
humidity  because  of  a  broken  gas  line.  Breakage  was  caused  by  intergrandular  corrosion 
caused  by  salt  spray,  which  v  eakims  the  tube  making  it  susceptible  to  mechanical  failure.  The 
tube  will  be  painted  in  production  to  prevent  corrosion.  Round  was  repaired  by  replacing  the 
exterior  gas  line  and  during  assembly  test  there  was  an  out  of  tolerance  condition  (low 
resistance  between  two  pins)  found  in  the  fuze.  Immersion  was  run  with  hardware  out  of 
tolerance  with  no  ac  .uonal  problems.  Later  analysis  determined  that  the  lower  resistance 
level  would  have  no  effect  on  flight  performance.  However,  the  round  was  not  flight  tested  be¬ 
cause  of  problems  with  target  availability. 

TC-1 

Missile  Round  received  x-ray  inspection,  launcher  assembly  tests,  rate  table  test,  and 
operational  test  prior  to  the  initiation  of  environmental  tests.  Round  was  subjected  to 
transportation  vibration,  36"  drop,  loose  cargo,  21"  drop,  mud  and  icing  environments. 
Following  each  environment,  except  mud,  the  round  was  exposed  to  a  visual  examination, 
operational  test,  disassembly,  rate  table  test,  assembly,  and  operational  test.  X-Ray 
inspection  was  performed  after  all  environments  except  mud.  After  the  mud  environment,  the 
round  received  a  visual  examination  and  operational  test.  The  round  passed  all  environments 
without  failure  and  was  flight  tested  as  PQTG-18. 

TC-2 

Missile  Round  received  x-ray  inspection,  launcher  assembly  test,  rate  table  test  and 
operational  test  prior  to  the  initiation  of  environmental  tests.  Round  wa°-  subjected  to 
transportation  vibration,  36"  drop,  loose  cargo,  21"  drop,  mud  and  icing.  Following  each 
environment,  except  mud,  the  round  was  subjected  to  a  visual  inspection,  operational  test, 
disassembly,  rate  table  test,  assembly,  and  operational  test.  X-Ray  inspection  was  performed 
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after  all  environments  except  mud.  After  the  mud  environment,  the  round  received  a  visual 
inspection  and  operational  test.  The  round  passed  aii  environments  without  failure  and  was 
flight  tested  as  PQTG-19. 

TC-3 

Missile  Round  received  x-ray  inspection,  launcher  assembly  test,  rate  table  test  and 
operational  test  prior  to  initiation  of  environmental  tests.  Round  was  subjected  to 
transportation  vibration.  36"  drop,  loose  cargo,  21"  drop,  mud,  icing.  Following  each 
environment  except  mud,  the  round  was  exposed  to  a  visual  inspection,  operational  test, 
disassembly,  rate  table  test,  assembly  and  operational  test.  During  the  disassembly/assembly 
operation  following  36"  drop,  the  water  block  cable  was  damaged  and  replaced;  this  was  not 
considered  an  environmental  failure.  X-ray  inspection  was  performed  after  ail  environments 
except  mud.  After  mud,  the  round  was  given  a  visual  inspection  and  operational  test.  The 
round  successfully  passed  all  environments  without  failure  and  was  flight  tested  as  PQTG-23. 
Although  general  indication  was  that  the  round  was  good,  the  flight  was  “no  tested”  for 
reliability  due  to  personnel  error/ mount  problem  that  induced  improper  lead  during  the 
prefire  operation. 

TC-4 

Weapon  Round  received  x-ray  inspection,  launcher  assembly  test,  rate  table  test  and 
operational  test  prior  to  initiation  of  environmental  tests.  Round  was  subjected  to 
transportation  vibration,  36"  drop,  loose  cargo,  21"  drop,  icing  and  mud.  After  each 
environment  except  icing,  the  round  received  a  visual  inspection,  operational  test, 
disassembly,  rate  table  test,  assembly  and  operational  test.  X-ray  inspection  was  performed 
after  all  environments  except  icing.  After  icing,  the  round  was  given  a  visual  inspection  and 
operational  test.  The  round  passed  all  environmental  tests  without  failure  and  was  flight  tested 
as  PQTG-16. 

TC-5 

Weapon  Round  received  x-ray  inspection,  launcher  assembly  test,  rate  table  test  and 
operational  test  prior  to  initiation  of  environmental  tests.  Round  was  subjected  to 
transportation  vibration,  36"  drop,  loose  cargo,  21"  drop,  icing  and  mud.  After  each 
environment  except  icing  the  round  received  a  visual  inspection,  operational  test,  disassembly, 
rate  table  test,  assembly  and  operational  test.  X-ray  inspection  was  performed  after  all 
environments  except  icing.  After  icing,  the  round  was  given  a  visual  inspection  and 


operational  test.  The  round  passed  all  environmental  tets  and  was  successfully  flight  tested  as 
PQTG-21. 

TC-6 

Weapon  Round  received  x-ray  inspection,  launcher  assembly  test,  rate  tabie  test  and 
operational  test  prior  to  initiation  of  the  environmental  tests.  Round  was  subjected  to 
transportation  vibration,  given  a  rate  table  test  and  then  an  operational  test.  A  vibration  was 
noticed  in  the  guidance  section;  analyses  later  determined  that  the  guidance  section  had  an 
unbalanced  gyro.  One  of  the  small  balance  weights  had  come  off  due  to  inadequate  bonding. 
The  gripstock  was  found  to  have  no  impulse  generator  output.  The  hardware  was  not  repaired 
and  put  back  into  environmental  testing. 

TCF-2 

Weapon  Round  received  visual  inspection,  rate  table  test  and  operational  test  prior  to  being 
innoculated  with  the  fungus  bacteria.  The  round  entered  the  environment  on  6  November 
1976  and  was  removed  on  5  January  1977.  After  completion  of  the  environment,  the  round 
was  visually  examined  operationally  tested.  The  gripstock  was  found  to  have  an  electrical 
malfunction,  not  related  to  the  environment.  The  missile  round  successfully  passed  the 
environment  and  was  flight  tested  as  PQTG-25. 

TH-1 

Missile  Round  received  x-ray  inspection,  launcher  assembly  test,  rate  table  test,  and 
operational  test  prior  to  initiation  of  the  environmental  tests.  Round  was  subjected  to 
transportation  vibration,  36"  drop,  loose  cargo,  21"  drop,  rain,  salt  fog,  humidity  and 
immersion.  Visual  examination  was  performed  after  each  environment.  After  transportation 
vibration,  the  round  was  given  an  operational  test,  disassembly,  rate  table  test,  assembly  and 
operational  test.  After  humidity,  cracks  were  observed  in  sight  frame  and  aft  sight  hinge 
inclosure  fixture  was  broken  from  sight  which  caused  a  boresight  problem.  After  immersion, 
the  round  was  given  x-ray  inspection  and  operational  test  and  found  to  have  a  broken  gas  line. 
The  line  was  replaced  and  the  round  was  fired  as  PQTG-26. 

TH-2 


Missile  Round  received  x-ray  inspection,  launcher  assembly  test,  rate  table  test  and  an 
operational  test  prior  to  initiation  of  environmental  tests.  The  round  was  subjected  to 
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transportation  vibration  and  received  a  visual  examination,  x-ray  inspection  and  an 
operational  test.  It  was  discovered  the  round  had  moved  in  the  launch  tube.  Damage  to  the 
missile  round  included  sheared  gas  line,  water  block  cable  broken,  blowout  disk  ruptured 
from  inside,  the  1R  window  broken  and  the  umbilical  retracted.  Analysis  indicated  the  cause 
of  failure  was  a  deformed  lock  spring  in  the  missile  detent.  N  o  design  c.iange  is  planned,  but  an 
inspection  point  will  be  added  during  production  to  prevent  recurrence.  The  hardware  was  not 
repaired. 

TH-3 

Missile  Round  received  x-ray  inspection,  launcher  assembly  test,  rate  table  test  and 
operational  test  prior  to  initiation  of  environmental  tests.  Round  was  subjected  to 
transportation  vibration,  36"  drop,  loose  cargo.  21"  drop,  rain,  humidity,  salt  fog  and 
immersion.  Visual  examination  was  performed  after  '".eh  environment  and  x-ray  inspection 
was  performed  after  humidity.  At  the  completion  of  immersion,  water  was  noted  in  the  launch 
tube.  Examination  revealed  that  the  castle  nut  had  cracked.  Cause  was  attributed  to  impure 
lexan  resulting  in  reduced  physical  properties.  Increased  quality  control  and  the  addition  of 
fiber  glass  in  the  area  of  the  castle  nut  is  being  incorporated  into  the  hardware  for  the 
production  prototype  test.  Water  was  detected  in  the  missile  dome;  extent  of  damage  was  not 
determined;  hardware  was  not  repaired. 

TH-4 

Weapon  Round  received  x-ray  inspection,  launcher  assembly  test,  rate  table  test  and 
operational  test  prior  to  initiation  of  environmental  tests.  Round  was  subjected  to 
transportation  vibration,  36"  drop,  loose  cargo,  21"  drop,  rain,  salt  fog,  humidity  and 
immersion.  Visual  examination  was  conducted  after  each  environment,  x-ray  inspection, 
disassembly,  rate  table  test,  assembly  and  operational  tests  were  conducted  after 
transportation  vibration,  36"  drop,  loose  cargo,  21"  drop  and  rain,  humidity  (no  x-ray)  and 
immersion.  After  humidity  the  sight  was  found  to  be  broken  and  the  gas  line  was  broken 
during  disassembly  operation;  line  was  replaced  and  round  continued.  The  round  was  flight 
tested  as  PQTG-24. 

TH-5 

Weapon  Round  received  x-ray  inspection,  launcher  assembly  test,  rate  table  test  and 
operational  test  prior  to  initiation  of  environmental  tests.  Round  was  subjected  to 
transportation  vibration,  36"  dror  '--ose  cargo,  21"  drop,  rain,  humidity,  salt  fog  and 


immersion.  Visual  examination  and  operating  tests  were  performed  after  each  environment; 
x-ray  inspection,  disassembly,  rate  table  test  and  assembly  performed  after  transportation 
vibration,  36"  drop,  loose  cargo,  2i"  drop  and  humidity  (no  x-ray).  After  immersion,  the 
round  was  found  to  have  a  plugged  cryostat  (no  gas  flow);  round  has  been  returned  to  the 
contractor  for  failure  analysis. 

TH-6 

Weapon  Round  received  x-ray  inspection,  launcher  assembly  test,  rate  table  test  and 
operational  test  prior  to  initiation  of  environmental  tests.  Round  was  subjected  to 
transportation  vibration  and  36"  drop  where  it  was  found  to  have  a  broken  gas  line;  line  was 
replaced  and  ur..t  put  back  in  test.  Round  then  received  loose  cargo,  21  "drop,  rain  and  salt  fog 
humidity  and  immersion.  Visual  examination  was  conducted  after  each  test;  x-ray  inspection 
and  operational  test  after  transportation  vibration,  36"  drop  and  immersion;  disassembly,  rate 
table  test  and  assembly  after  transportation  vibration  and  36"  drop.  After  humidity  the  aft 
sight  hinge  was  found  broken.  The  round  would  not  track  at  the  completion  of  immersion;  IR 
window  broken  and  water  beads  on  the  missile.  It  is  felt  that  the  IR  window  was  cracked 
during  handling  prior  to  immersion,  although  records  do  not  show  it.  This  round  was  not 
repaired. 

TH-7 

Weapon  Round  received  visual  examination  and  opeiational  test  prior  to  initiation  of 
environmental  tests.  Gripstock  serial  number  191  was  mistakenly  used  with  this  round  (it  has 
previously  failed  during  tests  of  TC-6)  and  all  gripstock  data  was  discounted.  The  missile 
round  received  transportation  vibration.  36"  drop,  loose  cargo,  21 "  drop,  rain  and  humidity. 
Visual  examination  was  made  after  each  environmental  and  an  operational  test  after 
humidity.  During  humidity,  the  castle  nut  separated  and  allowed  moisture  in  the  launch  tube. 
This  problem  resulted  from  the  poor  physical  properties  of  impure  lexan.  Corrective  action  is 
provided  by  increased  quality  control  and  the  addition  of  fiber  glass.  The  round  was  not 
repaired. 
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